Training sample processing method, device, medium and electronic equipment

Abstract

The present disclosure relates to a training sample processing method, device, medium and electronic device, the method comprising: performing sentence detection on the acquired annotated named entity text to obtain multiple sentences corresponding to the named entity text; classifying each of the sentences based on a preset classifier to obtain the classification corresponding to each of the sentences; determining the sentence vector of each of the sentences based on a vector representation model; for each of the sentences, determining the sample set to which the sentence belongs based on the sentence vector of the sentence and the classification vector of the classification corresponding to the sentence, wherein the sample sets are multiple, and the probabilities of sentences in different sample sets being used as training samples are different. Thus, training samples for training the named entity recognition model can be selectively determined from the annotated data, and the influence of samples with low annotation quality being directly used as training samples on the training efficiency of the named entity recognition model can be avoided to a certain extent.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing, and in particular, to a training sample processing method, device, medium, and electronic device. Background Art

[0002] Natural Language Processing (NLP) is a key area of ​​research in computer science and artificial intelligence. Entity labeling is a complex task within NLP, requiring the annotator to possess specialized knowledge and standardized annotation standards. Named Entity Recognition data contains a large number of entities, making it easy for annotators to miss entities during the labeling process.

[0003] The data annotated by the annotators is used to train the named entity recognition model. The accuracy of the annotated data directly affects the training accuracy and efficiency of the named entity recognition model. Summary of the Invention

[0004] The purpose of the present disclosure is to provide an accurate and effective training sample processing method, device, medium and electronic device to improve the training accuracy and efficiency of named entity recognition models.

[0005] In order to achieve the above objectives, the first aspect of the present disclosure provides a training sample processing method, the method comprising:

[0006] Performing sentence detection on the acquired annotated named entity text to obtain multiple sentences corresponding to the named entity text;

[0007] Classifying each of the sentences based on a preset classifier to obtain a classification corresponding to each of the sentences;

[0008] Determining a sentence vector for each of the sentences based on a vector representation model;

[0009] For each of the sentences, the sample set to which the sentence belongs is determined based on the sentence vector of the sentence and the classification vector of the classification corresponding to the sentence, wherein there are multiple sample sets, and the probabilities of sentences in different sample sets being used as training samples are different.

[0010] Optionally, the classification vector of each of the categories is determined by:

[0011] Obtaining a labeled sample, wherein the labeled sample includes a training text and labeled named entities in the training text;

[0012] Performing sentence detection on the training text to obtain a plurality of training sentences corresponding to the training text;

[0013] Determining a label vector and a classification corresponding to each of the training sentences;

[0014] According to the label vectors corresponding to the training sentences belonging to the same category, the classification vector of the category is determined.

[0015] Optionally, determining the label vector and classification corresponding to each training sentence includes:

[0016] Determining the classification corresponding to each of the training sentences based on a preset classifier;

[0017] For each of the training sentences, determining the sentence annotated named entity corresponding to the training sentence according to the annotated named entities in the training text;

[0018] Based on the vector representation model, vectors of each labeled character in the sentence labeled named entity corresponding to the training sentence are determined, and an average vector of the vectors of each labeled character in the training sentence is determined as the labeled vector corresponding to the training sentence.

[0019] Optionally, determining the sample set to which the sentence belongs based on the sentence vector of the sentence and the classification vector of the classification corresponding to the sentence includes:

[0020] determining a target distance between a sentence vector of the sentence and a classification vector of a classification corresponding to the sentence;

[0021] If the target distance is less than the distance threshold, the sample set to which the sentence belongs is determined to be a training sample set, and the probability corresponding to the training sample set is 1;

[0022] If the target distance is not less than the distance threshold, the sample set to which the sentence belongs is determined to be a candidate sample set, and the probability corresponding to the candidate sample set is greater than or equal to 0 and less than 1.

[0023] Optionally, determining the sentence vector of each sentence based on the vector representation model includes:

[0024] For each of the sentences, based on the vector representation model, a vector of each marked character in the sentence is determined, and an average vector of the vectors of the marked characters in the sentence is used as the sentence vector of the sentence.

[0025] Optionally, the vector representation model is implemented by training based on a BERT model using each labeled named entity text and unlabeled named entity text in the corpus as training data.

[0026] Optionally, the method further includes:

[0027] For each sample set, a target sentence is selected from the sentences in the sample set according to the probability corresponding to the sample set, and the target sentence and the annotation information of the target sentence are used as training samples for training a named entity recognition model.

[0028] According to a second aspect of the present disclosure, a training sample processing apparatus is provided, the apparatus comprising:

[0029] A detection module, configured to perform sentence detection on the acquired annotated named entity text to obtain a plurality of sentences corresponding to the named entity text;

[0030] A classification module, configured to classify each of the sentences based on a preset classifier to obtain a classification corresponding to each of the sentences;

[0031] A first determining module, configured to determine a sentence vector of each sentence based on a vector representation model;

[0032] The second determination module is used to determine, for each of the sentences, the sample set to which the sentence belongs based on the sentence vector of the sentence and the classification vector of the classification corresponding to the sentence, wherein there are multiple sample sets, and the probabilities of sentences in different sample sets being used as training samples are different.

[0033] According to a third aspect of the present disclosure, a non-temporary computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the steps of any one of the methods described in the first aspect are implemented.

[0034] According to a fourth aspect of the present disclosure, an electronic device is provided, including:

[0035] a memory having a computer program stored thereon;

[0036] A processor is used to execute the computer program in the memory to implement the steps of any one of the methods of the first aspect.

[0037] Through the above technical solution, after the named entity text is annotated, it is not directly used as a training sample for the named entity recognition model. Instead, the named entity text is subjected to sentence detection to determine the probability of each sentence it contains being used as a training sample for the named entity recognition model. In this way, the training samples for training the named entity recognition model can be selectively determined from the annotated data, which to a certain extent avoids the impact of directly using samples with low annotation quality as training samples on the training efficiency of the named entity recognition model. At the same time, it can ensure the accuracy of the trained named entity recognition model and improve the accuracy of named entity recognition.

[0038] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following detailed description, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings:

[0040] Figure 1 is a flowchart of a training sample processing method provided according to an embodiment of the present disclosure;

[0041] Figure 2 is a block diagram of a training sample processing apparatus provided according to an embodiment of the present disclosure;

[0042] Figure 3 is a block diagram of an electronic device according to an exemplary embodiment;

[0043] Figure 4 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0044] The following describes the specific embodiments of the present disclosure in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present disclosure and are not intended to limit the present disclosure.

[0045] Figure 1 As shown in FIG. 1 , it is a flow chart of a training sample processing method provided according to an embodiment of the present disclosure, as shown in FIG. Figure 1 As shown, the method may include:

[0046] In step 11, sentence detection is performed on the acquired annotated named entity text to obtain multiple sentences corresponding to the named entity text.

[0047] A text can be output from the named entity text in the corpus to be annotated and displayed to the user so that the user can annotate the text. In response to the user's annotation operation, the user's annotation information of the text is obtained, thereby obtaining the annotated named entity text, which contains the entity annotation information.

[0048] As an example, sentence segmentation detection can be performed based on punctuation marks in the named entity text. For example, a target punctuation mark for determining sentence segmentation can be pre-set, such as one or more of a period, a comma, and a semicolon. Thus, punctuation marks in the named entity text can be detected, and the named entity can be sentence segmented based on the target punctuation marks.

[0049] For example, the target punctuation mark is a period, and the annotated named entity text is represented as follows:

[0050] Born and raised in Anhui. Unemployed. Denies smoking, drinking, or drug use. No history of exposure to industrial toxins, dust, or radioactive materials.

[0051] By performing sentence segmentation detection on the named entity text based on the target punctuation, we can obtain six sentences: 1. Born and raised in Anhui. 2. Unemployed. 3. Denies smoking. 4. Denies drinking. 5. Denies drug addiction. 6. No history of exposure to industrial toxins, dust, or radioactive substances.

[0052] In step 12, each sentence is classified based on a preset classifier to obtain a classification corresponding to each sentence.

[0053] Among them, part of the corpus can be selected from the corpus, and the corpus can be subjected to sentence detection to obtain the sentences corresponding to the corpus, and then each sentence of the corpus can be classified and labeled to obtain the training corpus of the classifier. For example, the classification can include but is not limited to: growth history, occupational history, smoking history, drinking history, drug addiction, history of contact with epidemic water and epidemic areas, history of contact with harmful substances, etc. Therefore, the sentences of the corpus can be used as the input of the model, and the classification labels corresponding to the sentences can be used as the target output of the model to train the model to obtain the classifier. As an example, the classifier can be implemented based on an LSTM (Long Short-Term Memory) network model, and the training method of the network model can be trained using the training method commonly used in this field, which will not be repeated here.

[0054] In this step, each sentence corresponding to the named entity text can be input into the classifier to obtain the classification corresponding to each sentence. For example, the classification determined is as follows:

[0055] 1. Born and raised in Anhui. The corresponding categories are: Growth history;

[0056] 2. Unemployed. The corresponding categories are: occupational history;

[0057] 3. Denial of smoking history. The corresponding categories are: smoking history;

[0058] 4. Denial of drinking history. The corresponding categories are: drinking history;

[0059] 5. Denial of drug addiction. The corresponding categories are: drug addiction;

[0060] 6. No history of exposure to industrial toxins, dust, or radioactive substances. Corresponding classification: History of exposure to hazardous substances.

[0061] In step 13, a sentence vector of each sentence is determined based on the vector representation model.

[0062] For example, the vector representation model can be implemented by training based on the BERT model using the various labeled named entity texts and unlabeled named entity texts in the corpus as training data. For example, the vector representation model can be trained using the various corpora in the corpus as training data to improve the accuracy of the vector representation model. For example, pre-training the BERT model can be to randomly block some words in the training data and train the model based on the blocked training data and the original training data, or to train the model by predicting whether two training data are connected. The training process and loss determination can adopt the common methods in this field, which will not be described here.

[0063] In this embodiment, by using all texts in the corpus, including labeled named entity texts and unlabeled named entity texts, as training data to determine the vector representation model, the accuracy of the vector representation model in vector conversion can be improved, the accuracy of the features in the sentence vector can be improved, and accurate data support can be provided for the subsequent determination of the sample set to which the sentence belongs.

[0064] In step 14, for each sentence, the sample set to which the sentence belongs is determined based on the sentence vector of the sentence and the classification vector of the classification corresponding to the sentence, wherein there are multiple sample sets, and the probabilities of sentences in different sample sets being used as training samples are different.

[0065] When training a model, labeled entities in the annotated text are typically used as positive samples, while unlabeled entities are used as negative samples. However, in existing techniques, the annotated text is often used directly as training samples after it has been annotated. However, if unlabeled entities appear in the annotated text, these unlabeled entities will be used as negative samples to evaluate the model's prediction loss, affecting the accuracy of model training.

[0066] Therefore, in this embodiment, after the named entity text is annotated, each sentence therein can be analyzed, so that each sentence can be divided into corresponding sample sets, so that the annotated named entity text can be further analyzed before being used as training data for the model, avoiding the impact of directly using the annotated text as training data on model training. In addition, when analyzing the named entity text, each sentence in the same named entity text can be analyzed separately to improve the accuracy of sample selection. By dividing the sample sets into different groups, the possibility of some sentences being used as training samples can be reduced.

[0067] Therefore, through the above technical solution, after the named entity text is annotated, it is not directly used as a training sample for the named entity recognition model. Instead, the named entity text is subjected to sentence detection to determine the probability of each sentence it contains being used as a training sample for the named entity recognition model. In this way, the training samples for training the named entity recognition model can be selectively determined from the annotated data, which to a certain extent avoids the impact of directly using samples with low annotation quality as training samples on the training efficiency of the named entity recognition model. At the same time, it can ensure the accuracy of the trained named entity recognition model and improve the accuracy of named entity recognition.

[0068] In a possible embodiment, the classification vector of each classification may be determined as follows:

[0069] Obtain annotated samples, wherein the annotated samples include training text and annotated named entities in the training text. The training text may be annotated by multiple users to ensure comprehensiveness and accuracy of the training text annotations and avoid missing entities in the training text.

[0070] The training text is subjected to sentence detection to obtain a plurality of training sentences corresponding to the training text. The specific implementation of this step is similar to the above-mentioned determination of the sentences corresponding to the named entity text, and will not be repeated here.

[0071] Determine a label vector and a classification corresponding to each training sentence, wherein the label vector corresponding to the training sentence is used to represent features of each labeled named entity in the training sentence.

[0072] According to the label vectors corresponding to the training sentences belonging to the same category, the classification vector of the category is determined.

[0073] For example, the average vector of the label vectors corresponding to the training sentences belonging to the same category can be determined as the category vector of the category.

[0074] Therefore, we can pre-annotate part of the corpus to determine the classification vectors under different categories. The classification vectors are determined based on the comprehensive annotation of each entity in the annotated text, and can be used to characterize the corresponding annotation features under the category, providing data reference for the subsequent determination of the comprehensiveness of the annotation of each annotated sentence.

[0075] In a possible embodiment, an exemplary implementation of determining the label vector and classification corresponding to each training sentence is as follows. This step may include:

[0076] Based on a preset classifier, determine the classification corresponding to each of the training clauses. The method for determining the classifier has been described above. In this step, the training clause can be input into the classifier for classification to obtain the classification corresponding to the training clause.

[0077] For each of the training clauses, determine the clause-annotated named entity corresponding to the training clause according to the annotated named entities in the training text.

[0078] Exemplarily, for each training clause, the entities in the annotated named entities in the training text that belong to the training clause can be determined as the clause-annotated named entity. For example, the named entities in the determined training clause "No history of exposure to dust and radioactive substances" can be "dust, radioactive substances".

[0079] Based on the vector representation model, determine the vectors of each annotated character in the clause-annotated named entity corresponding to the training clause, and determine the average vector of the vectors of each annotated character in the training clause as the annotation vector corresponding to the training clause.

[0080] Continuing with the above example, after determining the clause-annotated named entity corresponding to the training clause, the vectors of each annotated character in the clause-annotated named entity can be further determined, that is, the vectors of 7 annotated characters such as "powder, dust, radioactivity, substance", etc. Then, the average vector of the vectors of these 7 annotated characters can be further used as the annotation vector of the training clause "No history of exposure to dust and radioactive substances". Thus, for each training clause, the annotation vector corresponding to the training clause can be determined to represent the entity features annotated in the training clause, regardless of the unannotated character part, so as to provide data support for determining the classification vectors under different classifications, obtain the annotation features under each classification, and facilitate the detection of the comprehensiveness of the subsequently annotated named entity text.

[0081] In a possible embodiment, the exemplary implementation manner of determining the clause vector of each clause based on the vector representation model in step 12 is as follows. This step may include:

[0082] For each of the clauses, based on the vector representation model, determine the vector of each annotated character in the clause, and use the average vector of the vectors of each annotated character in the clause as the clause vector of the clause.

[0083] Exemplarily, taking the clause "No history of exposure to industrial poisons, dust, and radioactive substances" as an example, if the entity annotated for this clause is "radioactive substances", then the character vectors of 5 annotated characters such as "radioactivity, substance" can be determined based on the vector representation model.

[0084] In this embodiment, the average vector of the vectors of the five labeled characters can be used as the sentence vector of the sentence. Thus, the features of the labeled characters in the sentence can be included in the sentence vector. Therefore, when the sentence vector is compared with the classification vector corresponding to the sentence, the difference between the labeling features corresponding to the sentence vector and the labeling features corresponding to the classification vector can be determined, thereby detecting whether the sentence is completely labeled, and providing data support for determining whether there are any missing entities in the sentence.

[0085] In a possible embodiment, an exemplary implementation of determining the sample set to which the sentence belongs based on the sentence vector of the sentence and the classification vector of the classification corresponding to the sentence is as follows. This step may include:

[0086] Determine a target distance between the sentence vector of the sentence and the classification vector of the classification corresponding to the sentence. The method of calculating the distance between vectors is prior art and will not be described in detail here.

[0087] If the target distance is less than the distance threshold, the sample set to which the sentence belongs is determined to be a training sample set, and the probability corresponding to the training sample set is 1.

[0088] The distance threshold can be set based on the actual usage scenario and is not limited in this disclosure. When the target distance is less than the distance threshold, it indicates that the vector features of the entities actually annotated in the sentence are similar to the features of the entities that should be annotated under the category corresponding to the sentence. In this case, it can be considered that all entities in the sentence have been fully annotated, and the sentence can be directly added to the training sample set.

[0089] If the target distance is not less than the distance threshold, the sample set to which the sentence belongs is determined to be a candidate sample set, and the probability corresponding to the candidate sample set is greater than or equal to 0 and less than 1.

[0090] The probability corresponding to the candidate sample set can be pre-set according to the application scenario. For example, if the training accuracy requirement of the named entity recognition model is high, the probability corresponding to the candidate sample set can be set to a smaller value, such as 0.1, to minimize the impact of samples with the risk of missing entity labels participating in the training on the named entity recognition model. If the training accuracy requirement of the named entity recognition model is average, the probability corresponding to the candidate sample set can be set to 0.2, which can reduce the impact of samples with the risk of missing entity labels participating in the training on the named entity recognition model to a certain extent, and at the same time reduce the amount of annotation required for training samples.

[0091] When the target distance is not less than the distance threshold, it indicates that the vector features of the entities actually labeled in the sentence are significantly different from the features of the entities that should be labeled under the classification corresponding to the sentence. In this case, it can be considered that the entities in the sentence are not fully labeled, and there may be a risk of missing entities. In this case, the sentence can be added to the candidate sample set.

[0092] The probability corresponding to the candidate sample set is greater than or equal to 0 and less than 1, meaning that not all sentences in the candidate sample set will be used to train the named entity recognition model, thereby avoiding the impact of missing labeled data on model training. Thus, the comprehensiveness of the labeled sentence can be tested based on the distance between the sentence vector corresponding to the sentence and the classification vector of its corresponding classification, thereby improving the reliability and accuracy of the training samples for the named entity recognition model.

[0093] In a possible embodiment, the method further includes:

[0094] For each sample set, a target sentence is selected from the sentences in the sample set according to the probability corresponding to the sample set, and the target sentence and the annotation information of the target sentence are used as training samples for training a named entity recognition model.

[0095] For example, the sample set includes S1 and S2, wherein the probability corresponding to the sample set S1 is pre-set to 100%, and the probability corresponding to the sample set S2 is pre-set to 20%. Then, all the sentences in the sample set S1 can be used as training samples for training the named entity recognition model, while for the sample set S2, 20% of the sentences contained therein are selected as training samples to train the named entity recognition model, thereby reducing the influence of the sentences in the sample set S2 on the training of the named entity recognition model. For example, when selecting 20% ​​of the sample set S2 as training samples, as an example, the selection can be made randomly. As another example, the first 20% can be selected in order from small to large according to the target distance corresponding to each sample in the sample set S2, thereby obtaining some training samples from the sample set.

[0096] Therefore, when determining the training samples of the named entity recognition model, different probabilities can be selected for sentences in different sample sets, so that for sentences with the risk of missing entity labels, the possibility of the sentence participating in the model training as a training sample can be reduced, avoiding the impact of directly using all samples with low labeling quality as training samples on the training efficiency of the named entity recognition model, and improving the accuracy and efficiency of model training.

[0097] The following is a comparison of the accuracy of a named entity recognition model trained based on sample selection using the disclosed solution and the accuracy of a named entity recognition model trained based on sample selection in the prior art, where α represents the probability corresponding to the candidate sample set:

[0098] Existing solutions 88.76％ 88.76％ 88.76％ This disclosure 92.31％(α＝0) 92.48％(α＝0.1) 91.14％(α＝0.2)

[0099] Therefore, through the above comparison, it can be seen that processing the training samples based on the above method disclosed in the present invention can effectively reduce the impact of incompletely labeled samples as training samples of the model on the accuracy of the model, improve the accuracy of the named entity recognition model, and then improve the accuracy of named entity recognition, thereby enhancing the user experience.

[0100] Based on the same inventive concept, the present disclosure also provides a training sample processing device, such as Figure 2 As shown, the device 10 includes:

[0101] The detection module 100 is used to perform sentence detection on the acquired annotated named entity text to obtain multiple sentences corresponding to the named entity text;

[0102] A classification module 200 is configured to classify each of the sentences based on a preset classifier to obtain a classification corresponding to each of the sentences;

[0103] A first determining module 300 is configured to determine a sentence vector of each sentence based on a vector representation model;

[0104] The second determination module 400 is used to determine, for each of the sentences, the sample set to which the sentence belongs based on the sentence vector of the sentence and the classification vector of the classification corresponding to the sentence, wherein there are multiple sample sets, and the probabilities of sentences in different sample sets being used as training samples are different.

[0105] Optionally, the classification vector of each of the categories is determined by a third determination module, and the third determination module includes:

[0106] An acquisition submodule, configured to acquire annotated samples, wherein the annotated samples include training text and annotated named entities in the training text;

[0107] A detection submodule, configured to perform sentence detection on the training text to obtain a plurality of training sentences corresponding to the training text;

[0108] A first determination submodule is used to determine the label vector and classification corresponding to each training sentence;

[0109] The second determination submodule is used to determine the classification vector of the classification based on the label vectors corresponding to the training sentences belonging to the same classification.

[0110] Optionally, the first determining submodule includes:

[0111] A third determination submodule is configured to determine a category corresponding to each of the training sentences based on a preset classifier;

[0112] A fourth determining submodule is configured to determine, for each training sentence, a sentence labeled named entity corresponding to the training sentence based on the labeled named entities in the training text;

[0113] The fifth determination submodule is used to determine the vectors of each labeled character in the sentence labeled named entity corresponding to the training sentence based on the vector representation model, and determine the average vector of the vectors of each labeled character in the training sentence as the labeled vector corresponding to the training sentence.

[0114] Optionally, the second determining module includes:

[0115] a sixth determining submodule, configured to determine a target distance between the sentence vector of the sentence and the classification vector of the classification corresponding to the sentence;

[0116] a seventh determination submodule, configured to determine, if the target distance is less than a distance threshold, that the sample set to which the sentence belongs is a training sample set, and the probability corresponding to the training sample set is 1;

[0117] An eighth determining submodule is configured to determine, if the target distance is not less than the distance threshold, that the sample set to which the sentence belongs is a candidate sample set, and the probability corresponding to the candidate sample set is greater than or equal to 0 and less than 1.

[0118] Optionally, the first determining module is configured to:

[0119] For each of the sentences, based on the vector representation model, a vector of each marked character in the sentence is determined, and an average vector of the vectors of the marked characters in the sentence is used as the sentence vector of the sentence.

[0120] Optionally, the vector representation model is implemented by training based on a BERT model using each labeled named entity text and unlabeled named entity text in the corpus as training data.

[0121] Optionally, the device further comprises:

[0122] The fourth determination module is configured to select a target sentence from the sentences in each sample set according to the probability corresponding to the sample set, and use the target sentence and its annotation information as training samples for training a named entity recognition model.

[0123] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0124] Figure 3 FIG. 7 is a block diagram of an electronic device 700 according to an exemplary embodiment. Figure 3 As shown, the electronic device 700 may include: a processor 701 , a memory 702 , and may further include one or more of a multimedia component 703 , an input / output (I / O) interface 704 , and a communication component 705 .

[0125] The processor 701 is used to control the overall operation of the electronic device 700 to complete all or part of the steps in the above-mentioned training sample processing method. The memory 702 is used to store various types of data to support the operation of the electronic device 700. For example, this data may include instructions for any application or method operating on the electronic device 700, as well as application-related data, such as contact information, sent and received messages, pictures, audio, video, etc. The memory 702 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The multimedia component 703 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may be further stored in the memory 702 or sent via the communication component 705. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 704 provides an interface between the processor 701 and other interface modules. The above-mentioned other interface modules may be a keyboard, a mouse, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 705 is used for wired or wireless communication between the electronic device 700 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IOT, eMTC, or other 5G, etc., or a combination of one or more thereof, is not limited here. Therefore, the corresponding communication component 705 may include: a Wi-Fi module, a Bluetooth module, an NFC module, etc.

[0126] In an exemplary embodiment, the electronic device 700 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to execute the above-mentioned training sample processing method.

[0127] In another exemplary embodiment, a computer-readable storage medium including program instructions is further provided. When the program instructions are executed by a processor, the steps of the above-described training sample processing method are implemented. For example, the computer-readable storage medium may be the aforementioned memory 702 including the program instructions. The program instructions may be executed by the processor 701 of the electronic device 700 to implement the above-described training sample processing method.

[0128] Figure 4 1 is a block diagram of an electronic device 1900 according to an exemplary embodiment. For example, the electronic device 1900 may be provided as a server. Figure 4 The electronic device 1900 includes a processor 1922, which may be one or more, and a memory 1932 for storing a computer program executable by the processor 1922. The computer program stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processor 1922 may be configured to execute the computer program to perform the above-mentioned training sample processing method.

[0129] In addition, the electronic device 1900 may further include a power supply component 1926 and a communication component 1950. The power supply component 1926 may be configured to perform power management of the electronic device 1900, and the communication component 1950 may be configured to implement communication of the electronic device 1900, for example, wired or wireless communication. In addition, the electronic device 1900 may further include an input / output (I / O) interface 1958. The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server 2003. TM , Mac OS X TM , Unix TM , Linux TM etc.

[0130] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided. When executed by a processor, the program instructions implement the steps of the aforementioned training sample processing method. For example, the non-transitory computer-readable storage medium may be the aforementioned memory 1932 including the program instructions. The program instructions may be executed by the processor 1922 of the electronic device 1900 to implement the aforementioned training sample processing method.

[0131] In another exemplary embodiment, a computer program product is further provided. The computer program product includes a computer program that can be executed by a programmable device, and the computer program has a code portion for executing the above-mentioned training sample processing method when executed by the programmable device.

[0132] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings. However, the present disclosure is not limited to the specific details of the above embodiments. Within the technical concept of the present disclosure, various simple modifications can be made to the technical solutions of the present disclosure, and these simple modifications all fall within the scope of protection of the present disclosure.

[0133] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.

[0134] In addition, the various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.

Claims

1. A training sample processing method, characterized in that: The method comprises: Perform sentence detection on the acquired annotated named entity text to obtain multiple sentences corresponding to the named entity text; Classifying each of the sentences based on a preset classifier to obtain a classification corresponding to each of the sentences; Determine a sentence vector for each of the sentences based on the vector representation model; For each of the sentences, according to the sentence vector of the sentence and the classification vector of the classification corresponding to the sentence, determine the sample set to which the sentence belongs, wherein there are multiple sample sets, and the probabilities of sentences in different sample sets being used as training samples are different; The classification vector of each classification is determined in the following manner: Obtaining a labeled sample, wherein the labeled sample includes a training text and labeled named entities in the training text; Perform sentence detection on the training text to obtain a plurality of training sentences corresponding to the training text; Determine the label vector and classification corresponding to each of the training sentences; Determine the classification vector of the classification according to the label vectors corresponding to each training sentence belonging to the same classification; The step of determining the sample set to which the sentence belongs according to the sentence vector of the sentence and the classification vector of the classification corresponding to the sentence includes: Determine a target distance between a sentence vector of the sentence and a classification vector of a classification corresponding to the sentence; If the target distance is less than the distance threshold, determine that the sample set to which the sentence belongs is a training sample set, and the probability corresponding to the training sample set is 1; If the target distance is not less than the distance threshold, the sample set to which the sentence belongs is determined to be a candidate sample set, and the probability corresponding to the candidate sample set is greater than or equal to 0 and the probability is less than 1.

2. The method according to claim 1, characterized in that The step of determining the label vector and the classification corresponding to each training sentence includes: Based on a preset classifier, determining the classification corresponding to each of the training sentences; For each of the training sentences, determining the sentence annotated named entity corresponding to the training sentence according to the annotated named entity in the training text; Based on the vector representation model, vectors of each labeled character in the sentence labeled named entity corresponding to the training sentence are determined, and an average vector of the vectors of each labeled character in the training sentence is determined as the labeled vector corresponding to the training sentence.

3. The method according to claim 1, characterized in that The determining of the sentence vector of each sentence based on the vector representation model includes: For each of the sentences, based on the vector representation model, a vector of each marked character in the sentence is determined, and an average vector of the vectors of the marked characters in the sentence is used as the sentence vector of the sentence.

4. The method according to any one of claims 1 to 3, characterized in that The vector representation model is implemented by training based on the BERT model using each labeled named entity text and unlabeled named entity text in the corpus as training data.

5. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: For each of the sample sets, a target sentence is selected from the sentences in the sample set according to the probability corresponding to the sample set, and the target sentence and the annotation information of the target sentence are used as training samples for training a named entity recognition model.

6. A training sample processing device, characterized in that: The device comprises: A detection module, used to perform sentence detection on the acquired annotated named entity text to obtain multiple sentences corresponding to the named entity text; A classification module, used to classify each of the sentences based on a preset classifier to obtain a classification corresponding to each of the sentences; A first determination module, configured to determine a sentence vector of each of the sentences based on a vector representation model; A second determination module is used to determine, for each of the sentences, the sample set to which the sentence belongs according to the sentence vector of the sentence and the classification vector of the classification corresponding to the sentence, wherein there are multiple sample sets, and the probabilities of sentences in different sample sets being used as training samples are different; The classification vector of each classification is determined by a third determination module, and the third determination module includes: An acquisition submodule, used for acquiring annotated samples, wherein the annotated samples include training text and annotated named entities in the training text; A detection submodule, used to perform sentence detection on the training text to obtain a plurality of training sentences corresponding to the training text; A first determination submodule is used to determine a label vector and a classification corresponding to each of the training sentences; The second determination submodule is used to determine the classification vector of the classification according to the label vectors corresponding to the training sentences belonging to the same classification; Wherein, the second determining module includes: A sixth determination submodule, used to determine a target distance between the sentence vector of the sentence and the classification vector of the classification corresponding to the sentence; A seventh determination submodule is used to determine that the sample set to which the sentence belongs is a training sample set if the target distance is less than a distance threshold, and the probability corresponding to the training sample set is 1; The eighth determination submodule is configured to determine, if the target distance is not less than the distance threshold, that the sample set to which the sentence belongs is a candidate sample set, and the probability corresponding to the candidate sample set is greater than or equal to 0 and less than 1.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 5 are implemented.

8. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 5.

Images