Corpus extension method and device and storage medium

Abstract

The invention discloses a corpus extension method, corpus extension equipment and a storage medium. The corpus extension method comprises the following steps: performing word segmentation on an original corpus to obtain original vocabularies; determining score influence factors matched with the candidate replacement vocabularies based on the word meaning quantity corresponding to the candidate replacement vocabularies and / or the original vocabularies; calculating a replacement recommendation score corresponding to the candidate replacement vocabulary by using the score influence factor, and selecting the candidate replacement vocabulary of which the replacement recommendation score meets a preset condition as a target replacement vocabulary corresponding to the original vocabulary; and replacing the original vocabularies in the original corpus with the target replacement vocabularies corresponding to the original vocabularies to obtain the extended corpus, so that score influence factors matched with different candidate replacement vocabularies can be accurately obtained, the calculation accuracy of replacement recommendation scores is improved, the selection accuracy of the target replacement vocabularies of the original vocabularies is further improved, and the user experience is improved. On the premise of increasing the corpus quantity, the situation that the extended corpus has semantic change relative to the original corpus is reduced, and the quality of the extended corpus is ensured.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a corpus expansion method, device and storage medium. Background Art

[0002] With the continuous development of artificial intelligence and the increasing amount of data, natural language processing technology has been widely used in all walks of life. The training effect of natural language processing models depends on corpus data, especially high-quality corpus data.

[0003] However, when establishing existing corpora, it is necessary to manually enter the corpus one by one, which not only wastes a lot of manpower and takes a long time. It is easy to miss corpus or the corpus is not rich, resulting in low corpus quality. Especially for certain feature fields (such as medical field, financial field, etc.), it is difficult to obtain corpus data of sufficient quantity and quality. Summary of the invention

[0004] In order to solve the above technical problems, the present application at least provides a corpus expansion method, device and storage medium.

[0005] In a first aspect, the present application provides a corpus expansion method, the method comprising: performing word segmentation processing on an original corpus to obtain an original vocabulary corresponding to the original corpus; and obtaining candidate replacement vocabulary corresponding to the original vocabulary; wherein the original vocabulary and the candidate replacement vocabulary are marked with a number of word senses, and the number of word senses refers to the number of meanings contained in the vocabulary; based on the number of word senses corresponding to the candidate replacement vocabulary and / or the original vocabulary, determining a score influencing factor matching the candidate replacement vocabulary; wherein the score influencing factor refers to a factor that affects the actual semantics of the corpus when the candidate replacement vocabulary is replaced with the original vocabulary in the corpus; using the score influencing factor to calculate the replacement recommendation score corresponding to the candidate replacement vocabulary, selecting the candidate replacement vocabulary whose replacement recommendation score meets preset conditions as the target replacement vocabulary corresponding to the original vocabulary; replacing the original vocabulary in the original corpus with the target replacement vocabulary corresponding to the original vocabulary, to obtain an expanded corpus.

[0006] In one embodiment, the scoring influencing factors include the lexical semantic similarity between the original vocabulary and the candidate replacement vocabulary and / or the sentence semantic similarity between the original corpus and the corpus after the original vocabulary in the original corpus is replaced with the candidate replacement vocabulary; based on the number of word senses corresponding to the candidate replacement vocabulary and / or the original vocabulary, the scoring influencing factors for matching with the candidate replacement vocabulary are determined, including: if the number of word senses corresponding to the candidate replacement vocabulary and / or the original vocabulary is less than or equal to a preset number threshold, the lexical semantic similarity is used as the scoring influencing factor for matching with the candidate replacement vocabulary; if the number of word senses corresponding to the candidate replacement vocabulary and / or the original vocabulary is greater than the preset number threshold, the lexical semantic similarity and the sentence semantic similarity are used as the scoring influencing factors for matching with the candidate replacement vocabulary.

[0007] In one embodiment, the scoring influencing factors for candidate replacement vocabulary matching include lexical semantic similarity and sentence semantic similarity; the replacement recommendation score corresponding to the candidate replacement vocabulary is calculated using the scoring influencing factors, including: mapping the lexical semantic similarity to obtain a first score, and mapping the sentence semantic similarity to obtain a second score; performing weighted summation on the first score and the second score, and using the weighted sum result as the replacement recommendation score corresponding to the candidate replacement vocabulary.

[0008] In one embodiment, selecting candidate replacement words whose replacement recommendation scores meet preset conditions as target replacement words corresponding to the original words includes: calculating the importance of each original word in the original corpus; setting a score threshold for each original word based on the importance of each original word; wherein the importance and the score threshold are positively correlated; for each original word, selecting candidate replacement words whose replacement recommendation scores are greater than the score threshold corresponding to each original word, to obtain the target replacement words corresponding to each original word.

[0009] In one embodiment, the method also includes: extracting the semantics corresponding to the original corpus and / or the extended corpus to obtain standard semantic features; generating corpus based on the standard semantic features to obtain a sentence conversion corpus different from the original corpus and / or the extended corpus; and using the sentence conversion corpus as a new original corpus and / or a new extended corpus.

[0010] In one embodiment, before performing word segmentation processing on the original corpus to obtain the original vocabulary corresponding to the original corpus, the method also includes: obtaining original structured data; extracting entities and / or attributes from the original structured data to obtain a full element set; and generating the original corpus based on the full element set.

[0011] In one embodiment, generating original corpus based on the full element set includes: adding entities or attributes in the full element set to a preset corpus template to obtain initial corpus; inputting the initial corpus into a pre-trained natural language model for natural language conversion to obtain original corpus output by the natural language model.

[0012] In one embodiment, the method also includes: using the original corpus and the extended corpus as model training corpus; performing named entity recognition on the model training corpus to obtain key entities; and performing dependency syntax analysis on the model training corpus to obtain grammatical dependencies between each word in the model training corpus; based on the grammatical dependencies between each word in the model training corpus, identifying the entity association relationship between each key entity; generating triples based on the entity association relationship between each key entity and each key entity to obtain a triple set; using the triple set to train the pre-trained language model to obtain a trained language model.

[0013] According to a second aspect of the present application, there is provided a corpus expansion device, which includes: a word segmentation module, which is used to perform word segmentation processing on an original corpus to obtain an original vocabulary corresponding to the original corpus; and, to obtain a candidate replacement vocabulary corresponding to the original vocabulary; wherein the original vocabulary and the candidate replacement vocabulary are marked with a number of word senses, and the number of word senses refers to the number of meanings contained in the vocabulary; a factor matching module, which is used to determine a score influencing factor that matches the candidate replacement vocabulary based on the number of word senses corresponding to the candidate replacement vocabulary and / or the original vocabulary; wherein the score influencing factor refers to a factor that affects the actual semantics of the corpus when the candidate replacement vocabulary is replaced with the original vocabulary in the corpus; a vocabulary selection module, which is used to calculate the replacement recommendation score corresponding to the candidate replacement vocabulary using the score influencing factor, and select the candidate replacement vocabulary whose replacement recommendation score meets a preset condition as the target replacement vocabulary corresponding to the original vocabulary; and a vocabulary replacement module, which is used to replace the original vocabulary in the original corpus with the target replacement vocabulary corresponding to the original vocabulary, to obtain an expanded corpus.

[0014] A third aspect of the present application provides an electronic device, including a memory and a processor, wherein the processor is used to execute program instructions stored in the memory to implement the above-mentioned corpus expansion method.

[0015] A fourth aspect of the present application provides a computer-readable storage medium having program instructions stored thereon, which implement the above-mentioned corpus expansion method when the program instructions are executed by a processor.

[0016] The above scheme obtains the original vocabulary corresponding to the original corpus by performing word segmentation processing on the original corpus; and obtains the candidate replacement vocabulary corresponding to the original vocabulary; wherein the original vocabulary and the candidate replacement vocabulary are marked with the number of word meanings; based on the number of word meanings corresponding to the candidate replacement vocabulary and / or the original vocabulary, the score influencing factors matching the candidate replacement vocabulary are determined; the replacement recommendation score corresponding to the candidate replacement vocabulary is calculated using the score influencing factors, and the candidate replacement vocabulary whose replacement recommendation score meets the preset conditions is selected as the target replacement vocabulary corresponding to the original vocabulary; the original vocabulary in the original corpus is replaced with the target replacement vocabulary corresponding to the original vocabulary to obtain the extended corpus, and the score influencing factors matching different candidate replacement vocabulary can be accurately obtained according to the number of word meanings corresponding to the candidate replacement vocabulary and / or the original vocabulary, thereby improving the calculation accuracy of the replacement recommendation score, and then improving the accuracy of the selection of the target replacement vocabulary of each original vocabulary, and reducing the semantic changes of the extended corpus relative to the original corpus on the premise of increasing the amount of corpus, thereby ensuring the quality of the extended corpus.

[0017] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0019] Figure 1 is a schematic diagram of a solution implementation environment shown in an exemplary embodiment of the present application; Figure 2 is a flow chart of a corpus expansion method shown in an exemplary embodiment of the present application; Figure 3 is a flowchart of an exemplary embodiment of the present application showing the use of original corpus and extended corpus to train a language model; Figure 4 is a block diagram of a corpus expansion device shown in an exemplary embodiment of the present application; Figure 5 is a schematic structural diagram of an electronic device shown in an exemplary embodiment of the present application; Figure 6 It is a schematic diagram of the structure of a computer-readable storage medium shown in an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0020] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0021] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0022] The term "and / or" in this article is only an association information describing the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the previous and next associated objects are in an "or" relationship. In addition, "many" in this article means two or more than two. In addition, the term "at least one" in this article means any combination of at least two of any one or more of a plurality of, for example, including at least one of A, B, and C, can mean including any one or more elements selected from the set consisting of A, B, and C.

[0023] The corpus expansion method provided in the embodiments of the present application is described below.

[0024] Please refer to Figure 1 , Figure 1 1 is a schematic diagram of a solution implementation environment shown in an exemplary embodiment of the present application. The solution implementation environment may include a terminal 110 and a server 120, and the terminal 110 and the server 120 are connected to each other for communication.

[0025] The number of the terminal 110 may be one or more. The terminal 110 may be a smart phone, a tablet computer, a notebook computer, a desktop computer, etc., but is not limited thereto.

[0026] Server 120 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), as well as big data and artificial intelligence platforms.

[0027] In one example, the server 120 may perform expansion processing on the original corpus obtained from the terminal 110 to obtain an expanded corpus. Of course, the server 120 may store the expanded corpus locally, transmit it back to the terminal 110, or transmit it to other terminals.

[0028] In one example, a client of a target application is installed and running in the terminal 110. For example, the target application may be an application that provides a corpus expansion function, and the target application is used to expand the original corpus to obtain the expanded corpus. The server 120 may be a background server of the target application, which is used to provide background services for the client of the target application.

[0029] In the corpus expansion method provided in the embodiment of the present application, the execution subject of each step can be the terminal 110, such as the client of the target application installed and running in the terminal 110, or the server 120, or the terminal 110 and the server 120 interact and cooperate to execute, that is, part of the steps of the method are executed by the terminal 110 and the other part of the steps are executed by the server 120.

[0030] See also Figure 2 , Figure 2 is a flowchart of a corpus expansion method shown in an exemplary embodiment of the present application. The corpus expansion method can be applied to Figure 1 It should be understood that the method can also be applied to other exemplary implementation environments and be specifically executed by devices in other implementation environments, and this embodiment does not limit the implementation environment to which the method is applicable.

[0031] like Figure 2 As shown, the corpus expansion method at least includes steps S210 to S240, which are described in detail as follows: Step S210: performing word segmentation processing on the original corpus to obtain the original vocabulary corresponding to the original corpus; and obtaining the candidate replacement vocabulary corresponding to the original vocabulary; wherein the original vocabulary and the candidate replacement vocabulary are marked with the number of word meanings, and the number of word meanings refers to the number of meanings contained in the vocabulary.

[0032] The original corpus refers to the text that needs to be expanded.

[0033] Perform word segmentation on the original corpus and use the words obtained by word segmentation as the original words.

[0034] The N-Gram model, maximum matching algorithm, Viterbi algorithm, and other algorithms can be used to perform word segmentation on the original corpus. This application does not limit the specific method of word segmentation.

[0035] For example, if the original corpus is "Product A high-performance smart phone", the original vocabulary obtained through word segmentation processing includes "Product A", "high-performance", and "smart phone".

[0036] In addition, candidate replacement words corresponding to the original words are obtained, and the original words and the candidate replacement words are marked with word meaning quantities.

[0037] Among them, the number of word meanings refers to the number of meanings contained in a word. For example, the meanings that "ink" may contain include: Meaning 1: liquid containing pigments or dyes, used for writing or painting; Meaning 2: human culture and knowledge, then the number of word meanings corresponding to "ink" is 2.

[0038] Specifically, a candidate replacement vocabulary set corresponding to the original vocabulary is obtained, and the candidate replacement vocabulary set contains one or more candidate replacement vocabulary.

[0039] It should be noted that generally, the original corpus corresponds to multiple original words, and each original word may correspond to the same set of candidate replacement words, or each original word may correspond to a different set of candidate replacement words.

[0040] For example, a vocabulary library is pre-built, and the vocabulary library contains multiple words.

[0041] The vocabulary library may be directly used as a set of candidate replacement words corresponding to each original word.

[0042] Alternatively, the similarity of vocabulary features between the current original vocabulary and each vocabulary in the vocabulary library is calculated, and the vocabulary in the vocabulary library whose vocabulary feature similarity is greater than a preset threshold is selected, or the vocabulary ranked in the top N after the vocabulary feature similarity is sorted in descending order is selected, and the selected vocabulary is used as the candidate replacement vocabulary corresponding to the current original vocabulary to obtain the candidate replacement vocabulary set corresponding to the current original vocabulary. By selecting candidate replacement vocabulary for each original vocabulary in the above manner, the candidate replacement vocabulary set corresponding to each original vocabulary is obtained.

[0043] Step S220: Determine the score influencing factors that match the candidate replacement vocabulary based on the number of word meanings corresponding to the candidate replacement vocabulary and / or the original vocabulary; wherein the score influencing factors refer to the factors that affect the actual semantics of the corpus when the candidate replacement vocabulary and the original vocabulary are replaced in the corpus.

[0044] For example, some of the factors affecting the score are given below: Factor 1: Lexical semantic similarity between the original word and the candidate replacement word; Factor 2: the semantic similarity between the original corpus and the corpus after the original words in the original corpus are replaced with the candidate replacement words; Factor 3: The difference between the weight of the original word and the weight of the candidate word; Factor 4: The difference between the emotional color expressed by the original word and the emotional color expressed by the candidate replacement words.

[0045] Among them, lexical semantic similarity can be obtained by calculating the vector distance between the feature vectors corresponding to two words; sentence semantic similarity can be obtained by calculating the vector distance between the feature vectors corresponding to the two corpora before and after replacement; the weight of meaning refers to the degree of semantics expressed by the vocabulary, such as "contempt" and "scorn", the semantics expressed by "contempt" is heavier than that of "scorn"; emotional color refers to the emotional tendency expressed by the vocabulary, for example, "achievement" is a commendatory word, "result" is a neutral word, and "consequence" is a derogatory word.

[0046] Of course, in addition to the above-mentioned scoring influencing factors, other scoring influencing factors may also be selected, and this application does not limit this.

[0047] According to the number of word senses corresponding to the candidate replacement vocabulary and / or the original vocabulary, a score influencing factor matching the candidate replacement vocabulary is determined.

[0048] That is, the scoring influencing factors that match the candidate replacement vocabulary can be determined based on the number of word senses corresponding to the candidate replacement vocabulary; the scoring influencing factors that match the candidate replacement vocabulary can also be determined based on the number of word senses corresponding to the original vocabulary; or the scoring influencing factors that match the candidate replacement vocabulary can be determined based on the number of word senses corresponding to the candidate replacement vocabulary and the original vocabulary. In this case, the maximum value of the number of word senses of the candidate replacement vocabulary and the number of word senses of the original vocabulary can be selected as the number of word senses used as a reference for the final matching of the scoring influencing factors.

[0049] For example, according to the number of word senses corresponding to the candidate replacement vocabulary and / or the original vocabulary, the number of scoring influencing factors to be considered is determined, and the corresponding number of scoring influencing factors are selected from the above-mentioned scoring influencing factor set to obtain the scoring influencing factors matching the candidate replacement vocabulary. Among them, if the number of word senses corresponding to the candidate replacement vocabulary and / or the original vocabulary is greater, it indicates that the probability that the semantics of the corpus will change after the candidate replacement vocabulary replaces the original vocabulary is higher. Therefore, the more word senses corresponding to the candidate replacement vocabulary and / or the original vocabulary, the more corresponding scoring influencing factors.

[0050] For another example, score influencing factors that need to be considered are set in advance for different word meaning quantity intervals, and score influencing factors matching the candidate replacement vocabulary are obtained according to the word meaning quantity intervals in which the word meaning quantity corresponding to the candidate replacement vocabulary and / or the original vocabulary is located. For example, the score influencing factors corresponding to the first quantity interval [0, 1] include factor one; the score influencing factors corresponding to the second quantity interval (1, 3] include factor two; and the score influencing factors corresponding to the third quantity interval (3, 5] include factor one, factor two, and factor three.

[0051] Step S230: Calculate the replacement recommendation scores corresponding to the candidate replacement words by using the score influencing factors, and select the candidate replacement words whose replacement recommendation scores meet the preset conditions as the target replacement words corresponding to the original words.

[0052] The replacement recommendation score represents the degree to which the candidate replacement word can replace the original word.

[0053] The higher the replacement recommendation score, the lower the possibility that the corpus obtained by replacing the candidate replacement vocabulary with the original vocabulary in the original corpus will have semantic changes. Conversely, the lower the replacement recommendation score, the higher the possibility that the corpus obtained by replacing the candidate replacement vocabulary with the original vocabulary in the original corpus will have semantic changes.

[0054] Calculate the impact value corresponding to the selected score influencing factor, and convert the impact value into a replacement recommendation score.

[0055] For example, the lexical semantic similarity between the original vocabulary and the candidate replacement vocabulary is calculated, and the lexical semantic similarity is converted into a replacement recommendation score. Specifically, the higher the lexical semantic similarity, the higher the replacement recommendation score, and the lower the lexical semantic similarity, the lower the replacement recommendation score.

[0056] For another example, the sentence semantic similarity between the original corpus and the corpus after the original words in the original corpus are replaced with candidate replacement words is calculated, and the sentence semantic similarity is converted into a replacement recommendation score. Specifically, the higher the sentence semantic similarity, the higher the replacement recommendation score, and the lower the sentence semantic similarity, the lower the replacement recommendation score.

[0057] For example, the difference between the significance of the original vocabulary expression and the significance of the candidate replacement vocabulary expression is calculated, and the difference between the significance is converted into a replacement recommendation score. Specifically, the smaller the difference between the significance, the higher the replacement recommendation score, and the larger the difference between the significance, the lower the replacement recommendation score.

[0058] For another example, the difference between the emotional color expressed by the original vocabulary and the emotional color expressed by the candidate replacement vocabulary is calculated, and the difference between the emotional colors is converted into a replacement recommendation score. Specifically, the smaller the difference between the emotional colors, the higher the replacement recommendation score, and the larger the difference between the emotional colors, the lower the replacement recommendation score.

[0059] For the candidate replacement words corresponding to the current original word, the replacement recommendation score corresponding to each candidate replacement word is calculated using the score influencing factors matched by each candidate replacement word, so as to select the candidate replacement words whose replacement recommendation scores meet the preset conditions as the target replacement words corresponding to the original word.

[0060] For example, a candidate replacement word with a replacement recommendation score greater than a preset threshold is selected as a target replacement word corresponding to the original word.

[0061] For another example, the top M candidate replacement words ranked in descending order of replacement recommendation scores are selected as target replacement words corresponding to the original words.

[0062] The preset conditions for judging the replacement recommendation score can be flexibly set according to actual conditions, and this application does not limit this.

[0063] Step S240: Replace the original vocabulary in the original corpus with the target replacement vocabulary corresponding to the original vocabulary to obtain the expanded corpus.

[0064] After obtaining the target replacement vocabulary corresponding to the original vocabulary, the original vocabulary and its corresponding target replacement vocabulary in the original corpus are replaced to obtain the extended corpus.

[0065] It should be noted that if the original vocabulary corresponds to multiple target replacement vocabulary, multiple vocabulary replacements are performed on the original corpus in sequence to obtain multiple extended corpora.

[0066] In the above manner, one or more original words in the original corpus are replaced to obtain an extended corpus corresponding to the original corpus. If multiple original words in the original corpus are replaced, only one original word may be replaced each time, or multiple original words may be replaced at the same time each time, which is not limited in this application.

[0067] Next, some embodiments of the present application are described in detail.

[0068] In some embodiments, the original corpus used in the present application may be manually written corpus, and / or corpus generated by a corpus generation neural network model, and / or corpus automatically generated based on a corpus template.

[0069] Exemplarily, the steps of generating the original corpus may include: acquiring original structured data; extracting entities and / or attributes from the original structured data to obtain a full element set; and generating the original corpus based on the full element set.

[0070] The original structured data may be any structured text data related to the corpus to be generated.

[0071] Entities or attributes are extracted from the original structured data to obtain a full element set. It should be noted that when entities and / or attributes are extracted from the original structured data, the relationship between the entities and / or attributes is also extracted to facilitate subsequent corpus generation.

[0072] For example, extract all attributes, entities and their relationships from the original structured data Draw to form a full element set ,in, Represents the i-th entity or attribute, where i ranges from 1 to n, and n is the number of entities and / or attributes extracted.

[0073] Generate the original corpus based on the full element set E.

[0074] For example, entities or attributes in the full element set are added to a preset corpus template to obtain initial corpus; the initial corpus is input into a pre-trained natural language model for natural language conversion to obtain the original corpus output by the natural language model.

[0075] In order to improve the standardization and consistency of corpus generation, a set of standardized preset corpus template sets T is constructed. Any preset corpus template in the preset corpus template set T A sentence structure is predefined, and the initial corpus is generated by combining the entities and attributes in the full element set E. .

[0076] Preset Corpus Template can be represented as a natural language structure containing placeholders, such as , by substituting the information in the full element set E into the preset corpus template , see the following formula 1 for the specific process:

[0077] Generated initial corpus As input to natural language models.

[0078] For example, for a product entity in a database, the generated corpus describing the product includes: the product is a smart phone with high performance.

[0079] Among them, the natural language model can be implemented based on network architectures such as a text-to-text transfer model (Text-To-Text Transfer Transformer, T5), a recurrent neural network (Recurrent Neural Network, RNN), or a Transformer model.

[0080] The training steps of the natural language model include: In the design of the natural language model, the input data consists of a natural language description generated based on the full element set E (the input in the application process is the initial corpus), denoted as , where each natural language description is a text description of a specific entity or attribute. The expected output data is the natural language target description corresponding to the input data X, denoted as Ensure that the natural language target description can accurately express the relevant entities, attributes, and relationships, and is semantically complete and syntactically accurate, so that the natural language model can learn high-quality language patterns.

[0081] In the training process of natural language models, the goal is to learn a mapping function , which is used to map input data X to expected output data Y. To achieve this goal, the natural language model can use the cross entropy loss function to measure the difference between the actual output data of the model and the expected output data. Specifically, the cross entropy loss function is defined as the following formula 2:

[0082] in represents the model parameters, is the tth word in the ith expected output text sequence in the expected output data, is the tth word in the actual output text sequence predicted by the natural language model, and it depends on Previously generated words and input , N is the total number of expected output text sequences in the expected output data, T is the total number of words contained in the text sequence, and L is the calculated model training loss.

[0083] Of course, in addition to the cross entropy loss function to calculate the model training loss, other loss functions can also be used to calculate the model training loss, which is not limited in this application.

[0084] By minimizing the model training loss, the natural language model can gradually learn how to generate natural language sequences that are close to the target description.

[0085] The training of the natural language model adopts the standard supervised learning method and is trained through the training set (X, Y). During the training process, the model generates sentences based on the given input and updates the parameters through back propagation. , to gradually optimize the quality of text generated by the natural language model.

[0086] Optionally, to ensure the text generation capability of the natural language model, it is necessary not only to detect the performance of the natural language model on the training set, but also to use the validation set to evaluate the performance of the natural language model to detect the text generation effect of the natural language model on unprocessed text data.

[0087] The performance evaluation of the validation set calculates the difference between the actual output text and the expected output text, and combines the language fluency, semantic accuracy, and text consistency of the actual output text to obtain the performance evaluation results of the natural language model. The performance evaluation results obtained from the validation set can be used to optimize the hyperparameters of the natural language model (such as learning rate and batch size), and can also help determine whether it is necessary to further adjust the preprocessing process of the input data, etc., in order to optimize the natural language model and obtain a trained natural language model.

[0088] Then, the initial corpus is input into the pre-trained natural language model for natural language conversion to obtain the original corpus output by the natural language model.

[0089] The trained natural language model can generate accurate natural language descriptions, ensuring good adaptability and usability in applications in specific vertical fields.

[0090] Of course, in addition to the original corpus generation method shown in the above embodiment, other methods can also be used to generate original corpus, for example, filling the entities or attributes in the full element set E into the preset corpus template, and directly using the filled result as the original corpus; for example, flexibly selecting entities or attributes in the full element set E, inputting them into the natural language generation neural network model, and using the natural language generation result as the original corpus. This application does not limit the specific method of obtaining the original corpus.

[0091] After obtaining the original corpus, it is necessary to expand the original corpus due to the lack of diversity of expression.

[0092] The target replacement vocabulary corresponding to the original vocabulary in the original corpus is determined, and the original vocabulary and the target replacement vocabulary are replaced to achieve corpus expansion.

[0093] Specifically, based on the number of word meanings corresponding to the candidate replacement vocabulary and / or the original vocabulary, the score influencing factors matching the candidate replacement vocabulary are determined, and the replacement recommendation scores corresponding to the candidate replacement vocabulary are calculated using the score influencing factors, and the candidate replacement vocabulary whose replacement recommendation scores meet preset conditions is selected as the target replacement vocabulary corresponding to the original vocabulary.

[0094] An example of a specific embodiment of the factors affecting the matching score is given below: The scoring influencing factors include the lexical semantic similarity between the original vocabulary and the candidate replacement vocabulary and / or the sentence semantic similarity between the original corpus and the corpus after the original vocabulary is replaced with the candidate replacement vocabulary; in step S220, based on the number of word senses corresponding to the candidate replacement vocabulary and / or the original vocabulary, the scoring influencing factors matching the candidate replacement vocabulary are determined, including: if the number of word senses corresponding to the candidate replacement vocabulary and / or the original vocabulary is less than or equal to a preset number threshold, the lexical semantic similarity is used as the scoring influencing factor for the candidate replacement vocabulary matching; if the number of word senses corresponding to the candidate replacement vocabulary and / or the original vocabulary is greater than the preset number threshold, the lexical semantic similarity and the sentence semantic similarity are used as the scoring influencing factors for the candidate replacement vocabulary matching.

[0095] For example, assuming that the preset quantity threshold is 1, if the number of word senses corresponding to the candidate replacement vocabulary or the original vocabulary is 1, the lexical semantic similarity between the candidate replacement vocabulary and the original vocabulary is calculated; if the number of word senses corresponding to the candidate replacement vocabulary or the original vocabulary is greater than 1, the sentence semantic similarity between the original corpus and the corpus after the original vocabulary in the original corpus is replaced with the candidate replacement vocabulary is calculated, and the lexical semantic similarity between the candidate replacement vocabulary and the original vocabulary is calculated.

[0096] Of course, the above embodiment is only an example of a specific application scenario. The selection method of the scoring influencing factors can be flexibly determined according to the actual application scenario, and this application does not limit this.

[0097] Based on the above embodiment, if the scoring influencing factors of the candidate replacement vocabulary matching include lexical semantic similarity and sentence semantic similarity, the scoring influencing factors are used to calculate the replacement recommendation score corresponding to the candidate replacement vocabulary, including: mapping the lexical semantic similarity to obtain a first score, and mapping the sentence semantic similarity to obtain a second score; performing weighted summation of the first score and the second score, and using the weighted sum result as the replacement recommendation score corresponding to the candidate replacement vocabulary.

[0098] Among them, the lexical semantic similarity is positively correlated with the first score, and the sentence semantic similarity is positively correlated with the second score.

[0099] The weight parameter used for weighted summation of the first score and the second score may be preset based on experience, or may be flexibly calculated based on specific circumstances.

[0100] For example, the number of word senses corresponding to the candidate replacement vocabulary, or the number of word senses corresponding to the original vocabulary, or the sum of the number of word senses corresponding to the candidate replacement vocabulary and the number of word senses corresponding to the original vocabulary is obtained to obtain the target number of word senses, and the weight parameter used for weighted summation of the first score and the second score is set according to the target number of word senses. For example, the larger the target number of word senses, the greater the probability of semantic changes in the original corpus after vocabulary replacement. Therefore, the larger the weight parameter corresponding to the second score is, so as to reduce the probability of actual semantic changes in the original corpus before and after vocabulary replacement.

[0101] An example of a specific embodiment of selecting a target replacement word is given below: In step S230, the candidate replacement words whose replacement recommendation scores meet the preset conditions are selected as the target replacement words corresponding to the original words, including: Step S231: Calculate the importance of each original word in the original corpus.

[0102] For example, by analyzing the role or function of each original word in the original corpus, the importance of each original word can be determined. For example, the original words can be classified into objects, subjects and predicates, and different classification results correspond to different importance.

[0103] For another example, by counting the frequencies of occurrence of each original word in all original corpora, the importance of each original word can be obtained.

[0104] For another example, the importance of each original word in the original corpus is identified by using a pre-trained neural network model for identifying the importance of words.

[0105] This application does not limit the specific calculation method of the word importance.

[0106] Step S232: Based on the importance of each original word, a scoring threshold is set for each original word; wherein the importance is positively correlated with the scoring threshold.

[0107] That is, the higher the importance of the original words, the higher the scoring threshold, and the lower the importance of the original words, the lower the scoring threshold.

[0108] Step S233: for each original word, select candidate replacement words whose replacement recommendation scores are greater than the score thresholds corresponding to the original words, and obtain target replacement words corresponding to the original words.

[0109] According to the importance of each original word in the original corpus, the scoring threshold corresponding to each original word is flexibly set, and then the target replacement vocabulary is selected for each original corpus. By raising the scoring threshold of important original words, the semantic accuracy of the expanded corpus obtained by the final expansion is guaranteed, and by reasonably setting the scoring threshold of unimportant original words, it is ensured that there are enough target replacement words, thereby increasing the number of expanded corpora.

[0110] Of course, in addition to the above embodiments, other methods may be used to select target replacement words, such as selecting candidate replacement words that rank high after sorting the replacement recommendation scores in descending order as target replacement words, which is not limited in the present application.

[0111] In some embodiments, in addition to corpus expansion through vocabulary replacement, the following can also be done: extracting the semantics corresponding to the original corpus and / or the expanded corpus to obtain standard semantic features; generating corpus based on the standard semantic features to obtain sentence conversion corpus that is different from the original corpus and / or the expanded corpus; and using the sentence conversion corpus as new original corpus and / or new expanded corpus.

[0112] Exemplarily, a sentence conversion model is pre-trained, and the original corpus and / or extended corpus are input into the sentence conversion model to obtain a sentence conversion corpus output by the sentence conversion model that has the same semantics as the original corpus and / or extended corpus but a different expression form, and the sentence conversion corpus is used as the new original corpus and / or the new extended corpus.

[0113] Among them, the sentence conversion model can be implemented based on a network architecture such as a BART (Bidirectional and Auto-Regressive Transformers) model, a recurrent neural network (RNN), or a Transformer model, and this application does not limit this.

[0114] Taking the BART model as an example, the BART model encodes the input corpus through a bidirectional encoder to obtain standard semantic features, and then decodes the standard semantic features through an autoregressive decoder to generate new original corpus and / or new extended corpus with different expressions. Since the BART model can capture the global and local semantic features of a sentence, the new corpus it generates not only has significant differences in syntactic structure, but also can generate more diverse natural language expressions while keeping the core semantics unchanged.

[0115] Through the above embodiments, the amount of the corpus finally expanded is further increased.

[0116] The following are some examples of specific application scenarios of the corpus: For example, see Figure 3 , Figure 3 FIG. 1 is a flowchart of an exemplary embodiment of the present application showing how to train a language model using original corpus and extended corpus. Figure 3 As shown, the specific steps include: Step S310: using the original corpus and the expanded corpus as model training corpus.

[0117] Step S320: performing named entity recognition on the model training corpus to obtain key entities; and performing dependency syntax analysis on the model training corpus to obtain grammatical dependency relationships between various words in the model training corpus.

[0118] The goal of named entity recognition is to identify words or phrases with practical meaning (such as names of people, places, organization names, etc.) from the model training corpus and associate them with other elements in the model training corpus.

[0119] By identifying key entities, we can provide necessary basic information for subsequent relationship extraction, ensuring that the key elements in the corpus can be captured.

[0120] Furthermore, dependency syntactic analysis is performed on the model training corpus to obtain the grammatical dependency relationship between each word in the model training corpus. Dependency syntactic analysis helps to capture the grammatical dependency structure between words in the text and can help understand the syntactic relationship between entities.

[0121] Among them, dependency syntactic analysis tools (such as SpaCy or Stanford Parser) can be used to perform dependency syntactic analysis on the model training corpus, which is not limited in this application.

[0122] Step S330: Based on the grammatical dependency relationship between each word in the model training corpus, the entity association relationship between each key entity is identified.

[0123] After identifying the key entities, the next step is to extract relations. Relation extraction is the process of identifying the relationships between entities. This application uses a dependency syntactic analysis tool to perform dependency syntactic analysis on the model training corpus, and constructs a dependency tree based on the grammatical dependency relationship between each word in the model training corpus obtained by analysis.

[0124] After obtaining the dependency tree of the model training corpus, a series of pre-defined rules are used to identify the entity association relationships between key entities.

[0125] For example, some relations may be represented by specific verbs or prepositional phrases, and dependency paths such as predicate verbs, objects or subjects in dependency relations can be used as the basis for rules, which can cover common relation patterns such as "belongs to", "consisting of", etc. By matching these dependency patterns and syntactic structures based on dependency trees, the relationships between key entities can be effectively identified.

[0126] For example, if "Company A is a subsidiary of Company B" appears in the model training corpus, the two key entities "Company A" and "Company B" are extracted, and the entity association relationship of "subsidiary" is matched based on the grammatical dependency relationship between each word.

[0127] Step S340: Generate triples based on each key entity and the entity association relationship between each key entity to obtain a triple set.

[0128] A triple can be represented as (entity, entity relationship, entity).

[0129] Step S350: train the pre-trained language model using the triple set to obtain a trained language model.

[0130] Triples provide the model with clear entity and relationship information, which can enhance its performance in tasks such as knowledge reasoning and question-answering systems.

[0131] Of course, in addition to using the corpus for language model training as described above, it can also be applied in other application scenarios, which is not limited in this application.

[0132] The corpus expansion method provided by the present application obtains the original vocabulary corresponding to the original corpus by performing word segmentation processing on the original corpus; and obtains the candidate replacement vocabulary corresponding to the original vocabulary; wherein the original vocabulary and the candidate replacement vocabulary are marked with the number of word meanings; based on the number of word meanings corresponding to the candidate replacement vocabulary and / or the original vocabulary, the score influencing factors matching the candidate replacement vocabulary are determined; the replacement recommendation score corresponding to the candidate replacement vocabulary is calculated using the score influencing factors, and the candidate replacement vocabulary whose replacement recommendation score meets the preset conditions is selected as the target replacement vocabulary corresponding to the original vocabulary; the original vocabulary in the original corpus is replaced with the target replacement vocabulary corresponding to the original vocabulary to obtain the extended corpus, and the score influencing factors matching different candidate replacement vocabulary can be accurately obtained according to the number of word meanings corresponding to the candidate replacement vocabulary and / or the original vocabulary, thereby improving the calculation accuracy of the replacement recommendation score, and further improving the accuracy of the selection of the target replacement vocabulary of each original vocabulary, and reducing the semantic changes of the extended corpus relative to the original corpus on the premise of increasing the amount of corpus, thereby ensuring the quality of the extended corpus.

[0133] Figure 4 FIG. 1 is a block diagram of a corpus expansion device shown in an exemplary embodiment of the present application. Figure 4 As shown, the exemplary corpus expansion device 400 includes: The word segmentation module 410 is used to perform word segmentation processing on the original corpus to obtain the original vocabulary corresponding to the original corpus; and obtain the candidate replacement vocabulary corresponding to the original vocabulary; wherein the original vocabulary and the candidate replacement vocabulary are marked with the number of word meanings, and the number of word meanings refers to the number of meanings contained in the vocabulary; A factor matching module 420 is used to determine a score influencing factor that matches the candidate replacement vocabulary based on the number of word meanings corresponding to the candidate replacement vocabulary and / or the original vocabulary; wherein the score influencing factor refers to a factor that affects the actual semantics of the corpus when the candidate replacement vocabulary is replaced with the original vocabulary in the corpus; The vocabulary selection module 430 is used to calculate the replacement recommendation score corresponding to the candidate replacement vocabulary by using the score influencing factors, and select the candidate replacement vocabulary whose replacement recommendation score meets the preset conditions as the target replacement vocabulary corresponding to the original vocabulary; The vocabulary replacement module 440 is used to replace the original vocabulary in the original corpus with the target replacement vocabulary corresponding to the original vocabulary to obtain the expanded corpus.

[0134] It should be noted that the corpus expansion device provided in the above embodiment and the corpus expansion method provided in the above embodiment belong to the same concept, wherein the specific manner in which each module and unit performs operations has been described in detail in the method embodiment, and will not be repeated here. In practical applications, the corpus expansion device provided in the above embodiment can allocate the above functions to different functional modules as needed, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above, and this is not limited here.

[0135] See also Figure 5 , Figure 5 1 is a schematic diagram of the structure of an embodiment of an electronic device of the present application. The electronic device 500 includes a memory 501 and a processor 502, and the processor 502 is used to execute program instructions stored in the memory 501 to implement the steps in any of the above-mentioned corpus expansion method embodiments. In a specific implementation scenario, the electronic device 500 may include but is not limited to: a microcomputer, a server, and in addition, the electronic device 500 may also include a mobile device such as a laptop computer and a tablet computer, which is not limited here.

[0136] Specifically, the processor 502 is used to control itself and the memory 501 to implement the steps in any of the above-mentioned corpus expansion method embodiments. The processor 502 can also be called a central processing unit (CPU). The processor 502 may be an integrated circuit chip with signal processing capabilities. The processor 502 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 502 can be implemented by an integrated circuit chip.

[0137] See also Figure 6 , Figure 6 The computer-readable storage medium 600 stores program instructions 610 that can be executed by a processor, and the program instructions 610 are used to implement the steps in any of the above-mentioned corpus expansion method embodiments.

[0138] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0139] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0140] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0141] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or part of the contribution to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program code.

Claims

1. A corpus expansion method, characterized in that: The method comprises: Performing word segmentation processing on the original corpus to obtain the original vocabulary corresponding to the original corpus; and obtaining candidate replacement vocabulary corresponding to the original vocabulary; wherein the original vocabulary and the candidate replacement vocabulary are marked with word meaning quantity, and the word meaning quantity refers to the number of meanings contained in the vocabulary; Based on the number of word meanings corresponding to the candidate replacement vocabulary and / or the original vocabulary, determining a score influencing factor matching the candidate replacement vocabulary; wherein the score influencing factor refers to a factor that affects the actual semantics of the corpus when the candidate replacement vocabulary and the original vocabulary are replaced in the corpus; Calculating the replacement recommendation score corresponding to the candidate replacement vocabulary using the score influencing factor, and selecting the candidate replacement vocabulary whose replacement recommendation score meets the preset conditions as the target replacement vocabulary corresponding to the original vocabulary; The original vocabulary in the original corpus is replaced with the target replacement vocabulary corresponding to the original vocabulary to obtain an expanded corpus.

2. The method according to claim 1, characterized in that The scoring influencing factors include the lexical semantic similarity between the original vocabulary and the candidate replacement vocabulary and / or the sentence semantic similarity between the original corpus and the corpus after the original vocabulary in the original corpus is replaced with the candidate replacement vocabulary; The step of determining the score influencing factors matching the candidate replacement vocabulary based on the number of word meanings corresponding to the candidate replacement vocabulary and / or the original vocabulary includes: If the number of word meanings corresponding to the candidate replacement vocabulary and / or the original vocabulary is less than or equal to a preset number threshold, the word semantic similarity is used as a scoring influencing factor for matching the candidate replacement vocabulary; If the number of word meanings corresponding to the candidate replacement vocabulary and / or the original vocabulary is greater than a preset number threshold, the word semantic similarity and the sentence semantic similarity are used as scoring influencing factors for matching the candidate replacement vocabulary.

3. The method according to claim 2, characterized in that The scoring influencing factors of the candidate replacement vocabulary matching include the vocabulary semantic similarity and the sentence semantic similarity; the step of calculating the replacement recommendation score corresponding to the candidate replacement vocabulary by using the scoring influencing factors includes: Obtaining a first score for the lexical semantic similarity mapping, and obtaining a second score for the sentence semantic similarity mapping; A weighted sum is performed on the first score and the second score, and the weighted sum result is used as the replacement recommendation score corresponding to the candidate replacement vocabulary.

4. The method according to claim 1, characterized in that The step of selecting a candidate replacement vocabulary whose replacement recommendation score satisfies a preset condition as a target replacement vocabulary corresponding to the original vocabulary includes: Calculating the importance of each original word in the original corpus; Based on the importance of each of the original words, respectively, a scoring threshold is set for each of the original words; wherein the importance is positively correlated with the scoring threshold; For each of the original words, candidate replacement words whose replacement recommendation scores are greater than the score thresholds corresponding to the original words are selected to obtain target replacement words corresponding to the original words.

5. The method according to claim 1, characterized in that The method further comprises: Extracting semantics corresponding to the original corpus and / or the extended corpus to obtain standard semantic features; Generating corpus based on the standard semantic features to obtain sentence conversion corpus different from the original corpus and / or the extended corpus; The sentence conversion corpus is used as a new original corpus and / or a new extended corpus.

6. The method according to claim 1, characterized in that Before performing word segmentation processing on the original corpus to obtain the original vocabulary corresponding to the original corpus, the method further includes: Get the original structured data; Extracting entities and / or attributes from the original structured data to obtain a full element set; An original corpus is generated based on the full element set.

7. The method according to claim 6, characterized in that The generating the original corpus based on the full element set includes: Adding entities or attributes in the full element set to a preset corpus template to obtain an initial corpus; The initial corpus is input into a pre-trained natural language model for natural language conversion to obtain the original corpus output by the natural language model.

8. The method according to claim 1, characterized in that The method further comprises: Using the original corpus and the extended corpus as model training corpus; Performing named entity recognition on the model training corpus to obtain key entities; and performing dependency syntax analysis on the model training corpus to obtain grammatical dependency relationships between various words in the model training corpus; Based on the grammatical dependency relationship between each word in the model training corpus, identifying the entity association relationship between each key entity; Generate triples based on the key entities and the entity association relationships between the key entities to obtain a triple set; The pre-trained language model is trained using the triple set to obtain a trained language model.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the processor is configured to execute program instructions stored in the memory to implement the steps in the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program instructions, and the program instructions can be executed by a processor to implement the steps in the method according to any one of claims 1 to 8.

Images