Text completion method, electronic device, and storage device
By determining the source of missing text content and using the text library to predict, automatic text completion is achieved, and the problems of inefficient and high cost of text completion in the existing technology are solved, and efficient and low-cost text completion is achieved.
Patent Information
- Application Number
- CN202011449650.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-12-09
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2040-12-09
AI Technical Summary
The prior art is inefficient and costly in the text completion process, and cannot effectively solve the problem of missing content caused by text transmission, storage, display and other links.
By obtaining the text to be completed, determining the source of its missing content, and using the corresponding text library to complete the completion prediction, we obtain candidate words in the missing position, thereby realizing automatic text completion, improving efficiency and reducing costs.
Text completion can be completed without manual intervention, which improves text completion efficiency and reduces costs, and can handle mixed granularity predictions such as text, words, and entities.
Smart Images

Figure CN113158648B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of natural language processing, and in particular to a text completion method, an electronic device, and a storage device. Background Art
[0002] With the development of information technology, text transmission through various networks such as wired / wireless has become one of the main means of communication in people's daily life and work. For example, text messages such as SMS and instant messaging are sent to friends and colleagues through mobile phones and tablets.
[0003] However, when sending, saving, and displaying text, some of its content may be missing due to various reasons. In addition, the missing content may be the core of the entire text. All these will have an adverse impact on the readability and usability of the text. At present, for missing content, manual completion is usually used to restore the missing content, which is inefficient and costly. In view of this, how to improve the efficiency of text completion and reduce the cost of text completion has become a topic of great research value. Summary of the invention
[0004] The main technical problem to be solved by the present application is to provide a text completion method, an electronic device, and a storage device, which can improve the efficiency of text completion and reduce the cost of text completion.
[0005] In order to solve the above-mentioned problem text, the first aspect of the present application provides a text completion method, including: obtaining a text to be completed, and determining a text library from which the missing content of the text to be completed originates; wherein the text to be completed includes at least one missing position; using the text library to perform completion prediction on the text to be completed, and obtaining at least one candidate word for the missing position; using the candidate words for each missing position, obtaining the complete text of the text to be completed.
[0006] In order to solve the above-mentioned problem text, the second aspect of the present application provides an electronic device, including a memory and a processor coupled to each other, the memory storing program instructions, and the processor being used to execute the program instructions to implement the text completion method in the above-mentioned first aspect.
[0007] In order to solve the above-mentioned problem text, the third aspect of the present application provides a storage device, which stores program instructions that can be executed by a processor, and the program instructions are used to implement the text completion method in the above-mentioned first aspect.
[0008] The above scheme obtains the text to be completed and determines the text library from which the missing content of the text to be completed originates, thereby using the text library to perform completion prediction on the text to be completed, obtaining at least one candidate word for the missing position, and then using the candidate words for each missing position to obtain the complete text of the text to be completed. Therefore, the missing content of the text to be completed can be completed without relying on manual labor, which can improve the efficiency of text completion and reduce the cost of text completion. In addition, since the missing content is determined to originate from the text library, the text library is used to perform completion prediction on the text to be completed, and at least one candidate word for the missing position is directly obtained, that is, the missing unknown is not limited to missing text, words or entities, which can be conducive to the prediction of mixed granularity such as text, words, and entities. BRIEF DESCRIPTION OF THE DRAWINGS
[0009] Figure 1 It is a flowchart of an embodiment of the text completion method of the present application;
[0010] Figure 2 It is a schematic diagram of the framework of an embodiment of the text completion method of the present application;
[0011] Figure 3 is a flow chart of an embodiment of a sample text acquisition process;
[0012] Figure 4 yes Figure 1 A schematic diagram of a flow chart of an embodiment of step S13;
[0013] Figure 5 is a state diagram of an embodiment of a word-by-word prediction process;
[0014] Figure 6 This is a flowchart of an embodiment of a first prediction network training process;
[0015] Figure 7 yes Figure 1 A schematic diagram of another embodiment of step S13;
[0016] Figure 8 is a state diagram of an embodiment of a prediction process using reference words;
[0017] Fig. 9 is a flow chart of an embodiment of a second prediction network training process;
[0018] Fig.10 yes Figure 1 A schematic flow chart of another embodiment of step S13;
[0019] Fig.11 It is a schematic diagram of the framework of an embodiment of a knowledge tree;
[0020] Fig.12is a state diagram of an embodiment of a fusion text acquisition process;
[0021] Fig.13 is a flow chart of an embodiment of a third prediction network training process;
[0022] Fig.14 It is a flowchart of another embodiment of the text completion method of the present application;
[0023] Fig.15 It is a flowchart of another embodiment of the method for completing the text of the present application;
[0024] Fig.16 It is a flowchart of another embodiment of the method for completing the text of the present application;
[0025] Fig.17 It is a schematic diagram of the framework of an embodiment of the electronic device of the present application;
[0026] Fig.18 It is a schematic diagram of the framework of an embodiment of the storage device of the present application. DETAILED DESCRIPTION
[0027] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.
[0028] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.
[0029] The terms "system" and "network" are often used interchangeably in this article. The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship. In addition, "many" in this article means two or more than two.
[0030] See also Figure 1 , Figure 1 This is a flowchart of an embodiment of the text completion method of the present application. Specifically, it may include the following steps:
[0031] Step S11: Obtain the text to be completed.
[0032] In the disclosed embodiment, the text to be completed includes at least one missing position, that is, the text to be completed may include 1 missing position, or may include multiple (such as 2, 3, etc.) missing positions, which are not limited here. For example, for a complete text "British medical journal The Lancet published online the results of the Phase II clinical trial of the XX vaccine of the Chinese Academy of Military Medical Sciences", the corresponding text to be completed may be "British () The Lancet published online the results of the Phase II clinical trial of the XX vaccine of the Chinese Academy of Military Medical Sciences", in the disclosed embodiment and the following other disclosed embodiments, unless otherwise specified, () indicates a missing position, and the above-mentioned text to be completed includes 1 missing position, or, the corresponding text to be completed may also be "() Country () The Lancet published online the results of the Phase II clinical trial of the XX vaccine of the Chinese Academy of Military Medical Sciences", that is, the text to be completed includes 2 missing positions, or, the corresponding text to be completed may also be "() Country () The Lancet published online the results of the Phase II clinical trial of the XX vaccine of the Chinese Academy of Military Medical Sciences", that is, the text to be completed includes 3 missing positions. Other situations can be deduced by analogy, and no examples are given one by one here.
[0033] It should be noted that each missing position may correspond to one missing word or multiple missing words (such as 2, 3, etc.). Taking the aforementioned complete text "British medical journal The Lancet published online the results of the Phase II clinical trial of the Chinese Academy of Military Medical Sciences" as an example, the corresponding full text to be completed can be "British () The Lancet published online the results of the Phase II clinical trial of the Chinese Academy of Military Medical Sciences XX vaccine", that is, the full text to be completed has 4 missing words at the missing position, or the full text to be completed can also be "British medical journal The Lancet published online the results of the Phase II clinical trial of the Chinese Academy of Military Medical Sciences XX vaccine", that is, the full text to be completed has 2 missing words at the missing position, or the full text to be completed can also be "() Chinese medical journal The Lancet published online the results of the Phase II clinical trial of the Chinese Academy of Military Medical Sciences XX vaccine", that is, the full text to be completed has 1 missing word at the missing position. Other situations can be deduced by analogy, and no examples are given here one by one.
[0034] In addition, the present disclosure and other disclosures described below do not limit the reasons for missing characters in the text to be completed. For example, the text to be completed may be missing characters due to network congestion, encoding errors, and other problems in the transmission, storage, and display of confidential texts; or, for example, in the transmission of confidential texts, names of places, people, and institutions often use special encoding or conversion methods, which may lead to missing characters.
[0035] Step S12: Determine the source of the missing content in the text to be completed.
[0036] In the embodiment of the present disclosure, the source situation includes any one of the following: unknown source, originating from a first text library, and originating from a second text library involving a preset knowledge field. Specifically, when the source situation includes originating from the first text library, it can be known that the missing content of the text to be completed comes from the first text library, but the exact knowledge field involved in the missing content cannot be determined. Taking the text to be completed "The World Intellectual Property Organization is headquartered in ()" as an example, it can be known that its missing content comes from an encyclopedia (such as Wikipedia, Baidu Encyclopedia and other online encyclopedias); and when the source situation includes originating from the second text library involving a preset knowledge field, it can be known that the missing content of the text to be completed comes from the second text library, and the second text library involves the preset knowledge field. Taking the text to be completed "As one of the representatives of the Vienna Classical School, () was appointed as a court musician in Salzburg in 1772" as an example, it can be known that its missing content comes from the second text library involving the knowledge field of classical music (such as the aforementioned Wikipedia, Baidu Encyclopedia and other online encyclopedias, or professional books related to classical music); and when the source situation includes unknown source, it is impossible to know the exact knowledge field from which the missing content comes, and it is also impossible to know the text library from which the missing content comes. The above examples are only possible situations in actual application, and do not limit the text to be completed, the first text library, the second text library or the preset knowledge field. They can be set according to the actual application situation, and will not be given one by one here.
[0037] In one implementation scenario, the complete text of the text to be completed is sent by the sender to the receiver. If the receiver has missing text during the receiving process due to the aforementioned reasons, the source of the missing content of the text to be completed can be determined based on the prior agreement between the sender and the receiver. For example, if the sender and the receiver have no prior agreement, the source of the missing content of the text to be completed can be determined as "unknown source"; or, for example, if the sender and the receiver have agreed in advance that the text does not exceed the scope of the encyclopedia, the source of the missing content of the text to be completed can be determined as "derived from the encyclopedia"; or, for example, if the sender and the receiver have agreed in advance that the text does not exceed the encyclopedia involving classical music, the source of the missing content of the text to be completed can be determined as "derived from the encyclopedia involving the field of classical music knowledge". Other situations can be deduced in the same way, and no examples are given here one by one.
[0038] In another implementation scenario, as mentioned above, the complete text of the text to be completed is sent by the sender to the receiver, and the receiver has missing text during the receiving process due to the aforementioned reasons. In this case, the source of the missing content of the text to be completed can also be determined based on the context of the historical conversation between the sender and the receiver. For example, if the historical conversation between the sender and the receiver does not involve a specific topic, the source of the missing content of the text to be completed can be determined as "unknown source"; or, for example, if the historical conversation between the sender and the receiver mainly involves ancient and modern Chinese and foreign figures, but is not limited to a specific field, the source of the missing content of the text to be completed can be determined as "derived from an encyclopedia"; or, for example, if the historical conversation between the sender and the receiver mainly involves representatives of various schools of classical music, the source of the missing content of the text to be completed can be determined as "derived from an encyclopedia involving the field of classical music knowledge." Other situations can be deduced by analogy, and no examples are given here one by one.
[0039] In another implementation scenario, after obtaining the text to be completed, the user can also be prompted to assist in determining the source of the missing content of the text to be completed. Specifically, the user can be prompted to select the text library from which the missing content of the text to be completed is derived, such as including: uncertain, encyclopedia, and other options, and when the user selects the "uncertain" option, the source of the missing content of the text to be completed is determined to be "unknown source", and after the user selects the text library from which the missing content of the text to be completed is derived, the user is further prompted to select the knowledge field involved in the missing content of the text to be completed, such as when the user selects the "encyclopedia" option, the user can be further prompted to select the knowledge field involved in the missing content of the text to be completed, such as including: uncertain, classical music, pop music and other options, and when the user selects "uncertain", the source of the missing content of the text to be completed is determined to be "derived from the encyclopedia", or when the user selects "classical music", the source of the missing content of the text to be completed is determined to be "derived from the encyclopedia involving the knowledge field of classical music". Other situations can be deduced by analogy, and no examples are given here one by one.
[0040] Step S13: Performing completion prediction on the text to be completed using a text prediction method that matches the source situation to obtain at least one candidate word for the missing position.
[0041] In an implementation scenario, when the source situation includes an unknown source, a preset number of default symbols can be added to each missing position of the text to be completed to obtain the text to be processed, and for each missing position, the text to be processed is predicted several times to obtain the predicted text of the default symbol at the corresponding sequence of the predicted number of times, and based on the predicted texts predicted several times, the candidate words of the missing position are obtained. In the above method, when the source situation includes an unknown source, a preset number of default symbols can be added to each missing position of the text to be completed to obtain the text to be processed, and for each missing position, the text to be processed is predicted several times to obtain the predicted text of the default symbol at the corresponding sequence of the predicted number of times, and then based on the predicted texts predicted several times, the candidate words of the default position are obtained, so that the text can be completed without relying on manual labor, which can improve the efficiency of text completion and reduce the cost of text completion. In addition, when the source is unknown, the text prediction is performed word by word at each missing position, which can be beneficial to improving the prediction accuracy, thereby improving the accuracy of text completion.
[0042] In a specific implementation scenario, the default symbol can be set according to actual application needs. For example, the default symbol can be set to [mask], which is not limited here.
[0043] In another specific implementation scenario, the preset value can be set according to actual application needs, such as 2, 3, 4, 5, etc., which is not limited here.
[0044] In another specific implementation scenario, in order to improve the prediction efficiency, the prediction of the text to be processed for each missing position can be performed by the first prediction network, that is, the text to be processed can be sent to the first prediction network, and finally the predicted text of the default symbol at the corresponding order of the prediction number can be obtained. For details, please refer to the relevant description in the following public embodiments, which will not be repeated here.
[0045] In another specific implementation scenario, taking the text to be completed "() Medical Journal "The Lancet" published online the results of the Phase II clinical trial of the XX vaccine of the Chinese Academy of Military Medical Sciences" as an example, after filling in 4 default symbols, the text to be processed can be obtained "[mask][mask][mask][mask] Medical Journal "The Lancet" published online the results of the Phase II clinical trial of the XX vaccine of the Chinese Academy of Military Medical Sciences". For the above-mentioned default positions, the first prediction can obtain the predicted text (such as, Britain, the United States, and France) at the default position (i.e., the first position) corresponding to the predicted number of times (i.e., 1 time), and the second prediction can obtain the predicted text (such as, country) at the default position (i.e., the second position) corresponding to the predicted number of times (i.e., 2 times). By analogy, the candidate words "Britain", "The United States", and "France" for the default position can be obtained. Other situations can be deduced by analogy, and examples are not given one by one here. The specific prediction process can refer to the relevant description in the following public embodiments, which will not be repeated here for the time being.
[0046] In an implementation scenario, when the source situation includes the situation that it is derived from the first text library, the first text library can be used to perform completion prediction on the text to be completed to obtain at least one candidate word for the missing position. In the above method, when the source includes the situation that it is derived from the first text library, the text to be completed is predicted by the first text library to obtain at least one candidate word for the missing position, so that the text can be completed without relying on manual labor, which can improve the efficiency of text completion and reduce the cost of text completion. In addition, when the missing content is derived from the first text library, the first text library is used to directly predict at least one candidate word for the missing position, which can be beneficial to further improve the efficiency of text completion. In addition, since the candidate words are directly predicted for the missing position, the missing position is not limited to missing text, words or entities, which can be beneficial to realize the prediction of mixed granularity such as text, words, and entities.
[0047] In a specific implementation scenario, in order to expand the scope of application, the first text library can specifically include text corpora that may be involved in the actual application process as much as possible, such as daily chats, or words, phrases, entities, etc. that may appear in various professional scenarios such as finance and music. For example, the first text library can specifically include corpora from online encyclopedias such as Baidu Encyclopedia and Wikipedia, so that the first text library can be applied to various business scenarios, greatly improving the scope of application.
[0048] In another specific implementation scenario, in order to improve the prediction efficiency, the completion prediction of the text to be completed can be performed by the second prediction network, that is, the text to be completed can be sent to the second prediction network, and finally at least one candidate word for the missing position is obtained. For details, please refer to the relevant description in the following public embodiments, which will not be repeated here.
[0049] In another specific implementation scenario, still taking the text to be completed "() Medical journal The Lancet published online the results of the Phase II clinical trial of the XX vaccine of the Chinese Academy of Military Medical Sciences" as an example, the first text library is used to complete the text to be completed and predict it, and at least one candidate word "Britain", "United States", "France" can be obtained for the missing position. Other situations can be deduced by analogy, and examples are not given one by one here. The specific prediction process can be referred to the relevant description in the following public embodiments, which will not be repeated here.
[0050] In an implementation scenario, when the source situation includes a second text library derived from a preset knowledge field, the knowledge graph corresponding to the preset knowledge field and the second text library can be used to perform a completion prediction on the text to be completed, and at least one candidate word for the missing position can be obtained. In the above method, when the source situation includes a second text library derived from a preset knowledge field, the knowledge graph corresponding to the preset knowledge field and the second text library are used to perform a completion prediction on the text to be completed, thereby obtaining at least one candidate word for the missing position, so that the text can be completed without relying on manual labor, which can improve the efficiency of text completion and reduce the cost of text completion. In addition, when the missing content originates from the second text library involving the preset knowledge field, on the one hand, the second text library is used to directly predict at least one candidate word for the missing position, which can be beneficial to further improve the efficiency of text completion, and on the other hand, the knowledge graph corresponding to the preset knowledge field is used to predict at least one candidate word for the missing position, which can be beneficial to improve the accuracy of the candidate words.
[0051] In a specific implementation scenario, in order to expand the scope of application, the second text library can specifically include text corpus that may be involved in the actual application process as much as possible, such as daily chats, or words, phrases, entities, etc. that may appear in various professional scenarios such as finance and music. For example, the second text library can specifically include corpus from online encyclopedias such as Baidu Encyclopedia and Wikipedia, so that the second text library can be applied to various business scenarios, greatly improving the scope of application.
[0052] In another specific implementation scenario, in order to improve the prediction efficiency, the completion prediction of the text to be completed can be performed by a third prediction network, that is, the text to be completed can be sent to the third prediction network, and finally at least one candidate word for the missing position is obtained. For details, please refer to the relevant description in the following public embodiments, which will not be repeated here.
[0053] In another specific implementation scenario, taking the text to be completed "The Lancet (UK) published online the results of the Phase II clinical trial of the XX vaccine of the Chinese Academy of Military Medical Sciences" as an example, the knowledge graph corresponding to the field of medical knowledge and the second text library can be used to predict the completion of the text to be completed, and at least one candidate word "medical journal", "journal", "newspaper" can be obtained for the missing position. Other situations can be deduced by analogy, and examples are not given one by one here. The specific prediction process can be referred to the relevant description in the following public embodiments, which will not be repeated here.
[0054] In addition, please refer to Figure 2 , Figure 2 Schematic diagram of the framework of an embodiment of the text completion method of this application. Figure 2 As shown, in order to improve the efficiency of completion prediction, when the source situation includes unknown source, the completion prediction can be performed using the first prediction network, and when the source situation includes originating from the first text library, the completion prediction can be performed using the second prediction network, or when the source situation includes originating from the second text library involving the preset knowledge field, the completion prediction can also be performed using the third prediction network. Therefore, in different source situations, different prediction networks can be used to perform completion prediction, which can be beneficial to expand the scope of application of text completion.
[0055] In an implementation scenario, in order to facilitate Figure 2 The text completion is performed within the framework of the method shown, the first text library and the second text library can be the same text library. As mentioned above, in order to expand the scope of application, the text library can specifically include as much text corpus as possible that may be involved in the actual application process, such as daily chat, or words, phrases, entities, etc. that may appear in various professional scenarios such as finance and music. For example, the text library can specifically include corpus from online encyclopedias such as Baidu Encyclopedia and Wikipedia, so that the text library can be applied to various business scenarios, greatly improving the scope of application.
[0056] In one implementation scenario, the first prediction network, the second prediction network, and the third prediction network may be obtained by respectively training different preset neural networks using different sample texts using different training methods. For example, the first prediction network may be obtained by training the first preset neural network using the first sample text using the first training method, while the second prediction network may be obtained by training the second preset neural network using the second sample text using the second training method, and the third prediction network may be obtained by training the third preset neural network using the third sample text using the third training method.
[0057] In another implementation scenario, in order to reduce the complexity of training, the first prediction network, the second prediction network, and the third prediction network can be obtained by training the same preset neural network using the same sample text using different training methods, that is, the first prediction network, the second prediction network, and the third prediction network can share sample text and preset neural network during the training process, thereby being conducive to reducing the complexity of training. The specific training methods of the first prediction network, the second prediction network, and the third prediction network can refer to the relevant descriptions in other disclosed embodiments of this application, which will not be repeated here.
[0058] It should be noted that the above-mentioned preset neural network can be set according to the actual application situation. For example, it can include but is not limited to: BERT (Bidirectional Encoder Representations from Transformers), ELMo, GPT (Generative Pre-Training), etc., which are not limited here.
[0059] Step S14: using the candidate words at each missing position to obtain the complete text of the text to be completed.
[0060] like Figure 2 As shown, after obtaining the candidate words for each missing position of the text to be completed, the candidate words for each missing position are further combined to perform joint completion prediction on the text to be completed, thereby obtaining the complete text of the text to be completed.
[0061] In an implementation scenario, a corresponding candidate word can be added to each missing position, so that several candidate texts for the text to be completed can be obtained, so that the final score of each candidate text can be obtained, and based on the final scores of the several candidate texts, a candidate text can be selected as the complete text of the text to be completed.
[0062] In a specific implementation scenario, the corresponding candidate words filled in the missing position are specifically the candidate words predicted at the missing position. Therefore, when the text to be completed includes n missing positions and each missing position corresponds to k candidate words predicted, the candidate texts to be completed have a total of k n. Taking the text to be completed "() Medical Journal The Lancet published online the results of the Phase II clinical trial of the XX vaccine of the () Military Medical Research Institute" as an example, the candidate words for the first missing position include: "Britain" and "United States", and the candidate words for the second missing position include: "China" and "Japan", then a corresponding candidate word can be added to each missing position, so that: "British Medical Journal The Lancet published online the results of the Phase II clinical trial of the XX vaccine of the Chinese Military Medical Research Institute", "American Medical Journal The Lancet published online the results of the Phase II clinical trial of the XX vaccine of the Chinese Military Medical Research Institute", "British Medical Journal The Lancet published online the results of the Phase II clinical trial of the XX vaccine of the Japanese Military Medical Research Institute", "American Medical Journal The Lancet published online the results of the Phase II clinical trial of the XX vaccine of the Japanese Military Medical Research Institute", a total of 4 candidate texts. Other situations can be deduced by analogy, and no examples are given here one by one.
[0063] In another specific implementation scenario, in order to improve the efficiency and accuracy of scoring candidate texts, several candidate texts can be sent to a preset scoring network to obtain the final scores of the corresponding candidate texts. The preset scoring network can be a statistical language network based on N-gram, which can specifically include but is not limited to: KenLM, SRILM, IRSTLM, BerkeleyLM, etc., which are not limited here. Taking N as 3 as an example, the final score can be expressed as:
[0064] P(w1,…,w n )=P(w1)*…*P(w n |w n-1 ,w n-2 )……(1)
[0065] In the above formula (1), w1,…,w n represents n words in the candidate text, specifically, w i represents the i-th word in the candidate text, and the right side of the equation is P(w1),…,P(w n |w n-1 ,w n-2) are predicted by the preset scoring network. Taking the aforementioned candidate text "The British medical journal The Lancet published online the results of the Phase II clinical trial of the XX vaccine of the Chinese Academy of Military Medical Sciences" as an example, the words in the candidate text can be: British, medical journal, Lancet, online, published, China, military, medicine, institute, XX vaccine, Phase II, clinical, trial, results. Before sending the candidate text to the preset scoring network, delimiters (such as spaces) can be used to distinguish the words in the candidate text. For example, the candidate text can be expressed as "The British medical journal The Lancet published online the results of the Phase II clinical trial of the XX vaccine of the Chinese Academy of Military Medical Sciences"; or, In order to adapt to various granularities such as characters, words, entities, etc., the words can be further segmented word by word based on the part-of-speech category of each word. For example, the words in the above candidate text can also be: British, National, Medical Journal, Lancet, Online, Published, China, National, Military, Medicine, Research Institute, XX Vaccine, Phase II, Clinical, Trial, Results. Before the candidate text is sent to the preset scoring network, a separator (such as a space) can be used to distinguish the various words in the candidate text. For example, the candidate text can be expressed as "The British Medical Journal Lancet published the results of the Phase II clinical trial of the Chinese Military Medical Research Institute of XX Vaccine". Other situations can be deduced by analogy, and examples are not given one by one here. The specific process of segmenting words word by word based on word categories can refer to the relevant description in the following disclosed embodiments of this application, which will not be repeated here.
[0066] In another specific implementation scenario, after obtaining the final scores of several candidate texts, the candidate text corresponding to the maximum final score can be selected as the complete text of the text to be completed. Still taking the above four candidate texts as an example, the above four candidate texts can be sent to the preset scoring network respectively to obtain the final scores of the candidate texts, and the candidate text corresponding to the maximum final score can be used as the complete text of the text to be completed. For example, when the final score of the candidate text "The British medical journal The Lancet published the results of the Phase II clinical trial of the XX vaccine of the Chinese Academy of Military Medical Sciences online" is the largest, it can be used as the complete text of the text to be completed. Other situations can be deduced by analogy, and no examples are given here one by one.
[0067] In another implementation scenario, in order to improve the accuracy of the final score, a corresponding candidate word can be added to each missing position to obtain several candidate texts of the text to be completed, and for each candidate text, the words in the candidate text are sorted in reverse order to obtain the reverse text of the candidate text, thereby obtaining the final score of the candidate text based on the first score of the candidate text and the second score of the reverse text, and then based on the final scores of the several candidate texts, a candidate text is selected as the complete text of the text to be completed. In the above method, a corresponding candidate word is added to each missing position to obtain several candidate texts of the text to be completed, and for each candidate text, the words in the candidate text are sorted in reverse order to obtain the reverse text of the candidate text, thereby obtaining the final score of the candidate text based on the first score of the candidate text and the second score of the reverse text, so that in the process of scoring the candidate text, the forward sequence and the reverse sequence of the candidate text can be comprehensively considered to score, so that it can be beneficial to improve the accuracy of the final score, and then in the subsequent process of obtaining the complete text based on the final score, it can be beneficial to improve the accuracy of the complete text.
[0068] In a specific implementation scenario, the corresponding candidate word filled in the missing position is specifically the candidate word predicted at the missing position. For details, please refer to the above-mentioned related description, which will not be repeated here.
[0069] In another specific implementation scenario, as mentioned above, in order to improve the efficiency and accuracy of scoring candidate texts, a first scoring network and a second scoring network can be pre-trained, so that the first score can be obtained by processing the candidate text using the first scoring network, and the second score can be obtained by processing the candidate text using the second scoring network. That is, for each candidate text, the candidate text can be sent to the first scoring network to obtain the first score, and the reverse text of the candidate text can be sent to the second scoring network to obtain the second score. In addition, as mentioned above, the first scoring network and the second scoring network can both be statistical language networks based on N-gram, which can specifically include but are not limited to: KenLM, SRILM, IRSTLM, BerkeleyLM, etc., which are not limited here. Taking N as 3 as an example, the first score can be obtained using the aforementioned related description, and the second score can be expressed as:
[0070] P(w1,…,w n )=P(w n )*…*P(w1|w2,w3)……(2)
[0071] In the above formula (1), w1,…,w n represents n words in the candidate text, specifically, w i represents the i-th word in the candidate text, and the right side of the equation P(wn ),…,P(w1|w2,w3), etc. are predicted by the second scoring network. Taking the aforementioned candidate text "British Medical Journal The Lancet published online the results of the Phase II clinical trial of the XX vaccine of the Chinese Military Medical Research Institute" as an example, the words in the candidate text can be: British, medical journal, Lancet, online, published, China, military, medical, research institute, XX vaccine, Phase II, clinical, trial, results, then the reverse text of the candidate text can be expressed as "results trial clinical Phase II XX vaccine research institute medical military China published online Lancet medical journal UK", before sending the reverse text to the second scoring network, the delimiters (such as spaces) can be used to distinguish the words in the reverse text, such as "results trial clinical Phase II XX vaccine research institute medical military China published online Lancet medical journal UK". Alternatively, in order to adapt to various granularities such as characters, words, entities, etc., the words can be further segmented word by word based on the part-of-speech category of each word. For example, the words in the above candidate text can also be respectively: English, Chinese, Medical Journal, Lancet, Online, Published, Chinese, Chinese, Military, Medical, Research Institute, XX Vaccine, Phase II, Clinical, Trial, Result, then the reverse text of the candidate text can be expressed as "Result Trial Clinical Phase II XX Vaccine Research Institute Medical Military Chinese Published Online Lancet Medical Journal Chinese English", before the reverse text is sent to the second scoring network, a separator (such as a space) can be used to distinguish the various words in the reverse text, such as "Result Trial Clinical Phase II XX Vaccine Research Institute Medical Military Chinese Published Online Lancet Medical Journal Chinese English". Other situations can be deduced by analogy, and examples are not given one by one here. The specific process of segmenting words word by word based on word categories can refer to the relevant description in the following disclosed embodiments of this application, which will not be repeated here.
[0072] In another specific implementation scenario, the final score can be obtained by weighting the first score and the second score using the first weight and the second weight respectively, and the first weight is not less than the second weight, for example, the first weight is 0.6 and the second weight is 0.4, or the first weight is 0.7 and the second weight is 0.3, which is not limited here. For the purpose of slogan description, the first weight can be recorded as λ and the second weight can be recorded as 1-λ, then the final score can be expressed as:
[0073] score = λg f (x)+(1-λ)g b (x)……(3)
[0074] In the above formula (3), score represents the final score of the candidate text x, g f (x) represents the first score of candidate text x, g b (x) represents the second score of candidate text x.
[0075] In another specific implementation scenario, after obtaining the final scores of several candidate texts, the candidate text corresponding to the maximum final score can be selected as the complete text of the text to be completed. For details, please refer to the above related description, which will not be repeated here.
[0076] The above scheme obtains the text to be completed, and the text to be completed includes at least one missing position, and determines the source of the missing content of the text to be completed, and the source situation includes any one of the following: unknown source, derived from the first text library, and derived from the second text library involving a preset knowledge field, so as to adopt a text prediction method matching the source situation to perform completion prediction on the text to be completed, and obtain at least one candidate word for the missing position, and then use the candidate words for each missing position to obtain the complete text of the text to be completed. Therefore, the missing content of the text to be completed can be completed without relying on manual labor, which can improve the efficiency of text completion and reduce the cost of text completion. In addition, since the text to be completed is predicted by a text prediction method matching the source situation, it can be beneficial to expand the scope of application of text completion.
[0077] As described in the above disclosed embodiments, the first prediction network, the second prediction network and the third prediction network can share sample texts and preset neural networks during the training process, so sample texts for subsequent training of the first prediction network, the second prediction network and the third prediction network can be pre-constructed, and on this basis, the samples can be used to train the first prediction network, the second prediction network and the third prediction network. Figure 3 , Figure 3 FIG. 1 is a flow chart of an embodiment of a sample text acquisition process. Figure 3 As shown, the following steps may be specifically included:
[0078] Step S31: segmenting and tagging the original text to obtain a number of words tagged with part-of-speech categories.
[0079] In an implementation scenario, the original text may be text related to the business scenario. For example, in a finance-related business scenario, the original text may include but is not limited to: finance news data, finance book data, etc.; or in a sports-related business scenario, the original text may include but is not limited to: sports news data, sports book data, etc. Other scenarios can be deduced by analogy, and examples are not given one by one here.
[0080] In another implementation scenario, in order to improve the efficiency of word segmentation and part-of-speech tagging, a word segmentation and part-of-speech tagging tool can be used to process the original text to obtain a number of words marked with part-of-speech categories. Specifically, the word segmentation and part-of-speech tagging tool can include but is not limited to: ICTCLAS, NLTK, Stanford NLP, etc., which are not limited here.
[0081] Taking the original text "China held the sports meeting for the first time in 2008" as an example, after the original text is processed by word segmentation and part-of-speech tagging, several words marked with part-of-speech categories can be obtained:
[0082]
[0083] The above letters are the part-of-speech tags for the words. For example, "ns" represents a place name, "nz" represents other named entities except common entities such as place names and personal names, "v" represents a verb, "m" represents a numeral, and "q" represents a measure word. In addition, words with relevant relationships can be combined. For example, the numeral "2008" and the measure word "year" can be combined into "2008 year". When the original text is in a certain situation, it can be deduced by analogy, and no more examples will be given here.
[0084] Step S32: Segment the words with the part-of-speech category of the preset category character by character, and select a preset proportion of words from the segmented words and the unsegmented words for default.
[0085] In an implementation scenario, the preset category can specifically be a place name. In this case, for the above original text "China held the sports meeting for the first time in 2008", the word "China" marked as a place name can be segmented character by character to obtain two characters, "zhong" and "guo". Other situations can be deduced by analogy, and no more examples will be given here.
[0086] It should be noted that in the process of obtaining the reverse text of the candidate text in the above disclosed embodiments, in order to adapt to various granularities such as characters, words, and entities, the candidate text can also be processed by word segmentation and part-of-speech tagging to obtain several words marked with part-of-speech categories, and the words with the part-of-speech category of the preset category can be segmented character by character. For specific details, reference can be made to the foregoing related descriptions in the disclosed embodiments of the present disclosure, and no further elaboration will be provided here. On this basis, the several segmented words can be sorted in reverse order to obtain the reverse text of the candidate text. Taking the foregoing candidate text "The British medical journal 'The Lancet' published online the results of the Phase II clinical trial of XX vaccine by the Chinese Academy of Military Medical Sciences" as an example, after the candidate text is processed by word segmentation and part-of-speech tagging, the following several words marked with part-of-speech categories can be obtained:
[0087]
[0088] The above letters are the part-of-speech tags for the words. For example, "vd" represents an adverbial verb, "n" represents a noun, and "nt" represents an institutional organization. On the basis that the preset category is a place name, the word "UK" marked as a place name can be segmented character by character into "ying" and "guo", and the word "China" marked as a place name can be segmented character by character into "zhong" and "guo". Other situations can be deduced by analogy, and no more examples will be given here.
[0089] In another implementation scenario, the preset ratio can be set according to the actual application situation. For example, in a business scenario with a large amount of missing content, the preset ratio can be set slightly larger, such as 30%, 35%, 40%, etc.; or, in a business scenario with relatively less missing content, the preset ratio can be set slightly smaller, such as 10%, 15%, 20%, etc. In addition, the preset ratio can also be set to a fixed value, such as 25%, which is not limited here. Still taking the original text "China held the sports meeting for the first time in 2008" as an example, the finally segmented words and the unsegmented words can be expressed as:
[0090] China held the sports meeting for the first time in 2008
[0091] That is, there are a total of 6 finally segmented words and unsegmented words. When the preset ratio is 1 / 3, 2 words can be selected for default among the above segmented words and unsegmented words. For example, "China" and "country" can be selected for default, or "in 2008" and "held" can also be selected for default, which is not limited here. For the original text and other preset ratio situations, it can be analogized in this way, and no more examples will be given here.
[0092] Step S33: Use the original text after default as the sample text, and use the positions of the default words as the sample missing positions of the sample text.
[0093] Still taking the above original text "China held the sports meeting for the first time in 2008" as an example, when "China" and "country" are selected for default, "()() held the sports meeting for the first time in 2008" can be used as the sample text, and use the position of the default word "China" as the sample missing position of the sample text, and use the position of the default word "country" as the sample missing position of the sample text. Other situations can be analogized in this way, and no more examples will be given here.
[0094] In one implementation scenario, for the convenience of subsequent training using the sample text, the original text after default and the default words can also be used as the sample text together. Still taking the above original text "China held the sports meeting for the first time in 2008" as an example, when "China" and "country" are selected for default, the original text after default "()() held the sports meeting for the first time in 2008" and the default words "China" and "country" can be used as the sample text together. Other situations can be analogized in this way, and no more examples will be given here.
[0095] Different from the aforementioned embodiment, by performing word segmentation and part-of-speech tagging on the original text, a number of words marked with part-of-speech categories are obtained, and the words with the part-of-speech category as the preset category are segmented word by word, so that a preset proportion of words are selected as default among the segmented words and the unsegmented words, and then the original text after the default is used as the sample text, and the position of the defaulted word is used as the sample missing position of the sample text. Therefore, it is possible to construct a sample text with missing content containing mixed granularities such as characters, words, entities, etc., which can help improve the adaptability of the prediction network obtained by subsequent training to the text to be completed with mixed granularities such as missing characters, words, entities, etc., and thus can help improve the accuracy of subsequent completion predictions.
[0096] See also Figure 4 , Figure 4 yes Figure 1 Schematic diagram of a flow chart of an embodiment of step S13 in the above embodiment. Specifically, the embodiment of the present disclosure is a flowchart of an embodiment of performing a completion prediction on a to-be-completed text when the source situation includes an unknown source. Specifically, the following steps may be included:
[0097] Step S41: Fill in a preset number of default symbols at each missing position of the text to be completed, to obtain the text to be processed.
[0098] In an implementation scenario, as described in the aforementioned disclosed embodiment, when the source situation includes an unknown source, the completion prediction can be performed using a first prediction network, and the first prediction network can be specifically obtained by training a preset neural network (such as BERT) using sample text. On this basis, the number of missing characters in the sample missing position in each sample text can be counted, and for several candidate values, the proportion of the number of missing characters less than the candidate value can be counted, so that in at least one candidate value whose proportion is greater than the preset percentage, the smallest candidate value is selected as the preset value. Specifically, the candidate values may include, but are not limited to: 1, 2, 3, 4, 5, 6, etc., which are not limited here. In addition, the preset percentage can be 90%, 92%, 95%, 97%, 99%, etc., which are not limited here. The above method counts the number of missing characters in the sample missing positions in each sample text, and for several candidate values, respectively, counts the proportion of the number of missing characters that is less than the candidate values, and then selects the smallest candidate value as the preset value among at least one candidate value whose proportion is greater than a preset percentage. This allows the preset value to cover most scenarios and reduce the number of default symbols as much as possible, which can help improve the efficiency of subsequent text prediction for each missing position.
[0099] In a specific implementation scenario, for N sample texts, statistics show that there are 20 sample missing positions in total, among which: The number of missing characters at the missing position of 1 sample is 1, the number of missing characters at the missing position of 3 samples is 2, the number of missing characters at the missing position of 3 samples is 3, the number of missing characters at the missing position of 12 samples is 4, and the number of missing characters at the missing position of 1 sample is 5. For candidate values 1, 2, 3, 4, and 5 respectively, the proportion of missing characters not greater than candidate value 1 is 1 / 20=5%, the proportion of missing characters not greater than candidate value 2 is 4 / 20=20%, the proportion of missing characters not greater than candidate value 3 is 7 / 20=35%, the proportion of missing characters not greater than candidate value 4 is 19 / 20=95%, and the proportion of missing characters not greater than candidate value 5 is 20 / 20=100%. When the preset percentage is 90%, the smallest candidate value 4 can be selected as the preset value from candidate value 4 with a corresponding proportion of 95% and candidate value 5 with a corresponding proportion of 100%. The same can be said for other situations, which will not be listed here one by one.
[0100] In another specific implementation scenario, as described in the aforementioned public embodiment, the default symbol can be set to [mask]. In addition, in order to facilitate the processing of the first prediction network, [CLS] and [SEP] can be added at the beginning and end of the text to be completed as the start mark and end mark respectively. Taking the aforementioned text to be completed "The World Intellectual Property Organization is headquartered in ()" as an example, it can be processed as "[CLS] The World Intellectual Property Organization is headquartered in [mask][mask][mask][mask][SEP]", and other situations can be deduced by analogy, and no examples are given here one by one.
[0101] In another specific implementation scenario, in order to facilitate the processing of the first prediction network, the text to be completed can also be segmented and POS tagged to obtain a number of words tagged with POS categories, and the words with POS categories as preset categories can be segmented word by word to obtain the text to be processed. Still taking the aforementioned text to be completed "The World Intellectual Property Organization is headquartered in ()" as an example, the relevant steps in the aforementioned public embodiment can be used to perform word segmentation, POS tagging and word segmentation, and finally obtain the following text to be processed:
[0102] [CLS]The World Intellectual Property Organization is headquartered in [mask][mask][mask][mask][SEP]
[0103] When the text to be completed is other text, the corresponding text to be processed can be obtained by analogy, and examples are not given one by one here.
[0104] In another implementation scenario, in order to facilitate the processing of the first prediction network, before the text to be processed is sent to the first prediction network, each word in the text to be processed can also be position-coded. Taking the above text to be completed "The World Intellectual Property Headquarters is located in ()" as an example, the text to be processed after position coding can be expressed as:
[0105]
[0106] When the text to be completed is other text, the position-coded text to be processed can be obtained by analogy, and examples are not given one by one here.
[0107] Step S42: For each missing position, the text to be processed is predicted several times to obtain the predicted text of the default symbol at the order corresponding to the number of predictions, and based on the predicted texts predicted several times, the candidate words of the missing position are obtained.
[0108] Specifically, when the i-th prediction is performed on the text to be processed, at least one predicted character of the default symbol at the i-th position and the predicted probability value of each predicted character can be obtained. On this basis, at least one predicted character of the default symbol at the i-th position can replace the default symbol at the i-th position to obtain at least one new text to be processed, and further determine whether the preset end condition is met at present. If the preset end condition is not met, i can be added by 1, and the step of performing the i-th prediction on the text to be processed and the subsequent steps can be re-executed. If the preset end condition is met, the candidate word for the missing position can be obtained based on the predicted probability value of each predicted character obtained most recently. In the above method, the text to be processed is predicted for the i-th time, at least one predicted character of the default symbol at the i-th position and the predicted probability value of each predicted character are obtained, and at least one predicted character of the default symbol at the i-th position replaces the default symbol at the i-th position respectively to obtain at least one new text to be processed. Therefore, if the preset end condition is not met, i is increased by 1, and the step of predicting the text to be processed for the i-th time and subsequent steps are re-executed. If the preset end condition is met, the candidate word of the missing position is obtained based on the predicted probability value of each predicted character in each of the texts to be processed that is newly obtained. Each prediction can rely on the previous prediction, thereby improving the correlation between the predicted characters obtained by word-by-word prediction, which is beneficial to improving the accuracy of the predicted characters.
[0109] In one implementation scenario, the preset end condition can be specifically set to any of the following: the predicted text is a preset end character, and i is not less than a preset value. Specifically, the preset end character can be set according to the actual application situation and is not limited here; in addition, the specific meaning of the preset value can refer to the relevant description in the aforementioned disclosed embodiment, which will not be repeated here.
[0110] In another implementation scenario, when predicting the text to be processed for the i-th time, specifically, the text to be processed can be fed into the first prediction network, so as to predict at least one predicted character of the default character at the i-th position and the predicted probability value of each predicted character. Specifically, when predicting the text to be processed for the i-th time, after feeding the text to be processed into the first prediction network, the semantic representation v of the default character at the i-th position can be obtained, and based on the semantic representation W of the preset vocabulary, the probability values of each character in the preset vocabulary can be obtained, which can be specifically expressed as:
[0111] p = softmax(v·W)……(4)
[0112] In the above formula (4), p represents the probability values of each character in the preset vocabulary, v represents the semantic representation of the default character at the i-th position, W represents the semantic representation of the preset vocabulary, · represents the dot product operation, and softmax represents the normalization process. In addition, the preset vocabulary includes the semantic representations of multiple (e.g., 30,000) common characters, which can be specifically obtained during the training process of the first prediction network. For example, BERT has semantic representations of approximately 30,000 different words. On this basis, at least one character (e.g., select 2 characters) can be selected as the predicted character at the i-th position in the order of decreasing probability values, and the corresponding probability values can be used as the predicted probability values of the predicted characters.
[0113] In yet another implementation scenario, please refer to Figure 5 , Figure 5 which is a schematic diagram of the state of an embodiment of the character-by-character prediction process. As shown in Figure 5, after the text to be completed "The World Intellectual Property Organization is headquartered in ()" is processed into the above text to be processed and fed into the first prediction network, when predicting the text to be processed for the first time, the semantic representation v of the default character [mask] at the first position can be obtained, and the dot product operation is performed on the semantic representation v and the semantic representations of each character of the preset words W (v1, v2, v3,..., v m ) to obtain the probability values of each character in the preset vocabulary, and the characters in the preset vocabulary are sorted (sort) in the order of decreasing probability values: Japan (i.e., W1), New York (i.e., W2), East (i.e., W3),..., North (i.e., W m ), and the characters ranked in the top preset positions (e.g., the top 2 positions) are selected, namely "Japan" and "New York" as the predicted characters of the default character [mask] at the first position during the first prediction, and the probability value of the predicted character "Japan" is used as the predicted probability value, and the probability value of the predicted character "New York" is used as the predicted probability value. On this basis, the predicted character "Japan" is used to replace the default character [mask] at the first position to obtain a new text to be processed, which can be denoted as the text to be processed 1 for the convenience of description:
[0114]
[0115] And replace the predicted text "Niu" with the default symbol [mask] at the first position, obtaining a new text to be processed, which can be denoted as the text to be processed 2 for the sake of convenience.
[0116]
[0117] When the preset end condition is not currently met, increment i by 1, so at this time i is 2, and so on. At the second prediction, the above two new texts to be processed can be sent into the first prediction network respectively. The text to be processed 1 undergoes a processing process similar to that of the first prediction, and the predicted texts "Nei" and "Ben" for the default symbol [mask] at the second position can be obtained at the second prediction. The text to be processed 2 undergoes a processing process similar to that of the first prediction, and the predicted texts "Yue" and "Ze" for the default symbol [mask] at the second position can be obtained at the second prediction. Further, replace the default symbol [mask] at the second position of the text to be processed 1 with the predicted texts "Nei" and "Ben" respectively, and 2 new texts to be processed can be obtained based on the text to be processed 1. Similarly, replace the default symbol [mask] at the second position of the text to be processed 2 with the predicted texts "Yue" and "Ze" respectively, and 2 new texts to be processed can be obtained based on the text to be processed 2. When the preset end condition is not currently met, increment i by 1, so at this time i is 3, and the above process can be re-executed. Finally, the predicted texts and predicted probability values obtained from each prediction are shown in Table 1. Table 1 is a summary table of the predicted texts and predicted probability values for each time. As shown in Table 1, based on the predicted text "Ri" obtained at the first prediction, the predicted texts "Nei" and "Ben" are obtained at the second prediction. Based on the predicted text "Nei" obtained at the second prediction, the predicted text "Wa" is obtained at the third prediction (on this basis, the fourth prediction ends and is not shown in Table 1). Based on the predicted text "Ben" obtained at the second prediction, the third prediction ends (i.e., the predicted text is empty). Based on the predicted text "Niu" obtained at the first prediction, the predicted texts "Yue" and "Ze" are obtained at the second prediction. Based on the predicted text "Yue" obtained at the second prediction, the third prediction ends (i.e., the predicted text is empty). Based on the predicted text "Ze" obtained at the second prediction, the predicted texts "Lan" and "Xi" are obtained at the third prediction (on this basis, the fourth prediction ends and is not shown in Table 1).
[0118] Table 1 Summary Table of Predicted Texts and Predicted Probability Values for Each Time
[0119]
[0120] It should be noted that the predicted text and predicted probability value shown in Table 1 is only one situation that may exist in the actual application process, and does not limit other situations that may exist in the actual application process. It can be set according to the actual application situation and is not limited here.
[0121] In another implementation scenario, when the preset end condition is met, the average probability value of the predicted probability values of each predicted word at the missing position can be calculated for each newly obtained text to be processed, and the text to be processed located at the previous preset position is selected in the order of the average probability value from large to small, and the combination of each predicted word at the missing position in the selected text to be processed is used as the candidate word for the missing position. The above method, by calculating the average probability value of the predicted probability value of each predicted word at the missing position for each newly obtained text to be processed, can be conducive to representing the overall accuracy of the predicted words in the text to be processed by the average probability value. On this basis, the text to be processed located at the previous preset position is selected in the order of the average probability value from large to small, and the combination of each predicted word at the missing position in the selected text to be processed is used as the candidate word for the missing position, so that the candidate word with greater overall accuracy can be selected, so it can be conducive to improving the accuracy of the candidate word, and then it can be conducive to improving the accuracy of the complete text finally obtained.
[0122] In a specific implementation scenario, the preset order can be set according to actual application needs. For example, in order to improve the speed of subsequent joint completion prediction using candidate words at various missing positions, the preset order can be set to a slightly smaller value, for example, it can be set to 2, 3, etc.; or, for example, to improve the robustness of subsequent joint completion using candidate words at various missing positions, the preset order can be set to a slightly larger value, for example, it can be set to 4, 5, etc., which is not limited here.
[0123] In another specific implementation scenario, still taking the aforementioned text to be supplemented in full "The World Intellectual Property Organization is headquartered in ()" as an example, please refer to Table 1 for reference. Among the predicted words at the missing positions in one text to be processed are "Geneva", and the average probability value of their predicted probability values is statistically 0.9. Among the predicted words at the missing positions in another text to be processed are "Japan", and the average probability value of their predicted probability values is statistically 0.8. Among the predicted words at the missing positions in another text to be processed are "New York", and the average probability value of their predicted probability values is statistically 0.875. Among the predicted words at the missing positions in another text to be processed are "New Zealand", and the average probability value of their predicted probability values is statistically 0.8. Among the predicted words at the missing positions in another text to be processed are "New Jersey", and the average probability value of their predicted probability values is statistically 0.78. Therefore, when the preset order is set to 2, the texts to be processed with the top 2 average probability values can be selected, and the combinations of the predicted words at the missing positions in the selected texts to be processed, namely "Geneva" and "New York", are used as the candidate words for the missing positions. Other situations can be inferred by analogy, and no more examples will be given here.
[0124] It should be noted that when the text to be supplemented in full includes multiple missing positions, the above method can be used to perform complementary prediction for each missing position respectively, and finally obtain the candidate words for each missing position. The specific process can refer to the foregoing description and will not be elaborated here.
[0125] Different from the foregoing embodiments, when the source situation includes an unknown source, a preset number of default symbols are respectively inserted at each missing position of the text to be supplemented in full, so as to obtain the text to be processed. For each missing position, the text to be processed is predicted several times to obtain the predicted words of the default symbols at the order positions corresponding to the number of prediction times. Then, based on the predicted words of several predictions, the candidate words for the default positions are obtained. Therefore, text completion can be performed without relying on manual labor, which can improve the efficiency of text completion and reduce the cost of text completion. In addition, when the source is unknown, predicting words character by character at each missing position can help improve the prediction accuracy, thereby helping to improve the accuracy of text completion.
[0126] Please refer to Figure 6 , Figure 6 is a schematic flowchart of an embodiment of the training process of the first prediction network. Specifically, it may include the following steps:
[0127] Step S61: At each sample missing position of the sample text, a preset number of default symbols are respectively inserted to obtain the sample text to be processed.
[0128] For details, please refer to the description of step S41 in the above-mentioned public embodiment, which will not be repeated here. In addition, the process of obtaining the sample text can be specifically referred to in the above-mentioned public embodiment and the appendix of the specification. Figure 3 The relevant description in will not be repeated here.
[0129] Step S62: For each missing position of the sample, the first prediction network is used to predict the sample text to be processed several times to obtain the sample prediction text and sample prediction probability value of the default symbol at the corresponding order of the prediction times.
[0130] Specifically, for each sample missing position, the first prediction network can be used to perform the i-th prediction on the sample to-be-processed text, obtain the sample prediction text of the default symbol at the i-th position and the sample prediction probability value of the sample prediction text, and replace the default symbol at the i-th position with the sample prediction text of the default symbol at the i-th position to obtain a new sample to-be-processed text. If the preset end condition is not met, i is added by 1, and the step of performing the i-th prediction on the sample to-be-processed text and subsequent steps are re-executed. If the preset end condition is met, the prediction of the current sample missing position can be terminated. For details, please refer to the relevant description of step S42 in the aforementioned public embodiment, which will not be repeated here.
[0131] Step S63: obtaining a first loss value of the first prediction network based on the sample prediction probability value of each sample predicted word in the sample candidate words at each sample missing position.
[0132] As described in the foregoing disclosed embodiments, in order to facilitate training using the sample text, the original text after defaulting and the defaulted words can be used as the sample text together. In addition, during the training process, since a preset number of default symbols are respectively filled in at each sample missing position of the sample text to obtain the sample text to be processed, several placeholder symbols (such as [PAD]) can be respectively filled in for each defaulted word, so that the total number of each defaulted word and the filled placeholder symbols is equal to the preset number. For example, still taking the foregoing original text "China held the sports meeting for the first time in 2008" as an example, in the case of selecting "中" and "国" for defaulting, the original text after defaulting "()()2008 held the sports meeting for the first time" and the defaulted words "中" and "国" can be used as the sample text together. Then, during the training process, when the preset number is set to 4, 4 default symbols can be respectively filled in at each default position in the sample text to obtain the sample text to be processed "[CLS][mask][mask][mask][mask][mask][mask][mask][mask]2008 held the sports meeting for the first time[SEP]", and 3 placeholder symbols are filled in for the defaulted word "中" to be converted to "中[PAD][PAD][PAD]". Similarly, 3 placeholder symbols are filled in for the defaulted word "国" to be converted to "国[PAD][PAD][PAD]". Other cases can be deduced by analogy and will not be exemplified one by one here.
[0133] In an implementation scenario, the first loss value can be calculated using the cross-entropy loss function, which can be specifically expressed as:
[0134]
[0135] In the above formula (5), M represents the number of sample missing positions in the sample text, N represents the foregoing preset number, and y ij represents the j-th character in the defaulted word corresponding to the i-th missing position, and p ij represents the sample prediction probability value of the sample predicted text predicted for the j-th time at the i-th missing position.
[0136] In addition, it should be noted that when using the above placeholder [PAD] to supplement the defaulted words, the preset end character in the preset end condition can be specifically set as the placeholder [PAD]. In the case of using other characters to supplement the defaulted words, the setting method of the preset end character can be deduced by analogy and will not be exemplified one by one here.
[0137] Step S64: Use the first loss value to adjust the network parameters of the first prediction network.
[0138] Specifically, the network parameters of the first prediction network can be adjusted using the first loss value by using methods such as stochastic gradient descent (SGD), batch gradient descent (BGD), and mini-batch gradient descent (MBGD). Batch gradient descent refers to using all samples to update parameters at each iteration; stochastic gradient descent refers to using one sample to update parameters at each iteration; and mini-batch gradient descent refers to using a batch of samples to update parameters at each iteration, which will not be repeated here.
[0139] Different from the above-mentioned embodiment, a preset number of default symbols are respectively added to each sample missing position of the sample text, thereby obtaining a sample text to be processed, and for each sample missing position, the sample text to be processed is predicted several times to obtain the sample predicted text and sample predicted probability value of the default symbol at the corresponding order of the prediction times, and based on the sample predicted probability value of each sample predicted text in the sample candidate words at each sample missing position, the first loss value of the first prediction network is obtained, thereby adjusting the network parameters of the first prediction network based on the first loss value. Therefore, performing word prediction at each sample missing position and adjusting the network parameters of the first prediction network based on the first loss value obtained by statistics can help improve the prediction accuracy of the first prediction network.
[0140] See also Figure 7 , Figure 7 yes Figure 1 Specifically, the disclosed embodiment is a flowchart of an embodiment of performing completion prediction on the to-be-completed text when the source situation includes originating from the first text library.
[0141] Specifically, the following steps may be included:
[0142] Step S71: perform semantic extraction on the text to be completed to obtain individual semantic representations of each missing position.
[0143] In an implementation scenario, in order to facilitate semantic extraction, a default symbol can be added to each missing position of the text to be completed to obtain the text to be processed, so as to perform semantic extraction on the text to be processed, and obtain the individual semantic representation of each default symbol as the individual semantic representation of the missing position where the default symbol is located. In the above method, a default symbol is added to each missing position of the text to be completed to obtain the text to be processed, and on this basis, semantic extraction is performed on the text to be processed to obtain the individual semantic representation of each default symbol, and the individual semantic representation of the default symbol is used as the individual semantic representation of the missing position where the default symbol is located. Therefore, this can be beneficial to the subsequent prediction of candidate words for the missing position based on the individual semantic representation of a single default symbol. Therefore, the missing position is not limited to missing text, words or entities, which can be beneficial to the prediction of mixed granularities such as text, words, and entities.
[0144] In a specific implementation scenario, the specific setting method of the default symbol can refer to the relevant description in the aforementioned disclosed embodiment, for example, it can be set to [mask], which is not limited here.
[0145] In another specific implementation scenario, [CLS] and [SEP] can be added at the beginning and end of the text to be completed as the start mark and end mark respectively. Taking the aforementioned text to be completed "The headquarters of the World Intellectual Property Organization is located in ()" as an example, it can be processed as "[CLS] The headquarters of the World Intellectual Property Organization is located in [mask][SEP]", and other situations can be deduced by analogy, and no examples are given here one by one.
[0146] In another specific implementation scenario, in order to facilitate the prediction of mixed granularities such as text, words, and entities, the text to be completed can be further segmented and tagged with parts of speech to obtain a number of words tagged with part-of-speech categories, and the words with part-of-speech categories as preset categories can be segmented word by word to obtain the text to be processed. The specific methods of word segmentation and part-of-speech tagging can refer to the relevant description in the aforementioned public embodiments, which will not be repeated here. In addition, the preset category can be set to a place name, which can be specifically referred to the relevant description in the aforementioned public embodiments, which will not be repeated here.
[0147] Still taking the aforementioned text to be completed "The World Intellectual Property Organization is headquartered in ()" as an example, the relevant steps in the aforementioned public embodiment can be used to fill in the default symbol, start mark and end mark, as well as word segmentation, part-of-speech tagging and word-by-word segmentation, and finally obtain the following text to be processed:
[0148] [CLS] The World Intellectual Property Organization is headquartered in [mask][SEP]
[0149] When the text to be completed is other text, the corresponding text to be processed can be obtained by analogy, and examples are not given one by one here.
[0150] In an implementation scenario, as described in the aforementioned disclosed embodiment, when the source situation includes originating from the first text library, the completion prediction can be performed using a second prediction network, and the second prediction network can be specifically obtained by training a preset neural network (such as BERT) using sample text. On this basis, the text to be completed can be sent to the second prediction network, so that the individual semantic representation of each missing position can be obtained. In addition, the training process of the second prediction network can be specifically referred to the following disclosed embodiment, which will not be repeated here.
[0151] In a specific implementation scenario, in order to facilitate the processing of the second prediction network, before the above-mentioned text to be processed is sent to the second prediction network, each word in the text to be processed can also be position-coded. Taking the above-mentioned text to be completed "The World Intellectual Property Headquarters is located in ()" as an example, the text to be processed after position coding can be expressed as:
[0152]
[0153] When the text to be completed is other text, the position-coded text to be processed can be obtained by analogy, and examples are not given one by one here.
[0154] Step S72: For each missing position, using the individual semantic representation of the missing position and the word semantic representations of each reference word, at least one candidate word for the missing position is obtained.
[0155] In the embodiment of the present disclosure, the first text library includes at least one reference text, and the reference text includes at least one reference word. The configuration of the first text library can be specifically referred to in the related description in the above disclosed embodiment, which will not be repeated here.
[0156] In an implementation scenario, at least one reference text may be segmented and POS tagged to obtain a number of words tagged with POS categories, and words with POS categories of preset categories may be segmented word by word, thereby obtaining a number of reference words using the segmented words and the unsegmented words, and then semantic extraction may be performed on the reference words to obtain semantic representations of the reference words. The above method, by segmenting and POS tagged at least one reference text to obtain a number of words tagged with POS categories, and words with POS categories of preset categories may be segmented word by word, thereby obtaining a number of reference words using the segmented words and the unsegmented words, and then it may be helpful to obtain reference words of mixed granularity including text, words, entities, etc., and it may be further helpful to realize the subsequent prediction of mixed granularity including text, words, entities, etc.
[0157] In a specific implementation scenario, the specific process of word segmentation, part-of-speech tagging, and word-by-word segmentation can refer to the relevant description in the aforementioned public embodiment, which will not be repeated here. In addition, the preset category can be set according to actual application needs, for example, it can be set to a place name, which can be specifically referred to the relevant description in the aforementioned public embodiment, which will not be repeated here.
[0158] In another specific implementation scenario, in order to further construct reference words with mixed granularity including text, words, and entities, the words with a part-of-speech category of a preset category can be segmented word by word. For the words segmented word by word, the words before segmentation and the text obtained after segmentation can be used as reference words. Taking the word "Beijing" as an example, its part-of-speech category is a place name. When the preset category is a place name, the word "Beijing" can be segmented word by word to obtain the text "North" and the text "Beijing". Therefore, the word "Beijing" before segmentation and the text "North" and "Beijing" obtained after segmentation can be used as reference words. Other situations can be deduced by analogy, and examples are not given one by one here.
[0159] In another specific implementation scenario, a preset word vector training tool (eg, word2vec, glove, etc.) may be used to perform word vector training on the reference words to extract the word semantic representation of the reference words.
[0160] In another specific implementation scenario, it is also possible to select words with a higher frequency of occurrence than a preset frequency from the segmented words and the unsegmented words to obtain a number of reference words. Specifically, the frequency of occurrence refers to the frequency of occurrence in the first text library. For example, according to statistics, there are a total of 100,000 words in the first text library, of which the word "Beijing" appears 100 times, so the frequency of occurrence of the word "Beijing" is 0.1%, and the word "gap" only appears once, so the frequency of occurrence of the word "gap" is 0.001%, and other situations can be deduced by analogy, and examples are not given one by one here. In addition, the preset frequency can be set according to actual application needs, for example, it can be set to 0.01%, 0.05%, etc., which is not limited here. In the above manner, in the segmented words and the unsegmented words, words with a higher frequency of occurrence than a preset frequency are selected to obtain a number of reference words, which can further streamline the scale of reference words.
[0161] In yet another specific implementation scenario, it is also possible to eliminate words whose part-of-speech categories meet the preset elimination conditions from the segmented words and unsegmented words, resulting in a number of reference words. Specifically, the preset elimination conditions can be set to include: any one of the part-of-speech categories being stop words or special symbols. Stop words can be function words in human language that have no actual meaning. For example, 'the', 'is', etc. in English, or, for example, 'um', 'ah', etc. in Chinese. In addition, special symbols can include but are not limited to: punctuation marks (such as the comma '、', etc.), unit symbols (such as kilograms 'kg', etc.), numbering symbols (such as ①, etc.), tab characters, currency symbols, etc., which are not limited here. By the above method, eliminating words whose part-of-speech categories meet the preset elimination conditions from the segmented words and unsegmented words to obtain a number of reference words can further streamline the scale of the reference words.
[0162] In one implementation scenario, for each missing position, the similarity between the individual semantic representation of the missing position and the word semantic representations of each reference word can be utilized. Thus, the reference words ranked in the top preset order can be selected in descending order of similarity as the candidate words for the missing position. Specifically, the similarity between the individual semantic representation and the word semantic representation can be the cosine similarity. In addition, the preset order can be set according to actual application needs. For example, to improve the speed of subsequent joint completion prediction using the candidate words for each missing position, the preset order can be set slightly smaller, such as 2, 3, etc.; or, for example, to improve the robustness of subsequent joint completion using the candidate words for each missing position, the preset order can be set slightly larger, such as 4, 5, etc., which are not limited here.
[0163] In another implementation scenario, using the individual semantic representation and the word semantic representations of each reference word, the predicted probability values of each reference word can be obtained. Specifically, the predicted probability value of a reference word can represent the likelihood that the word missing at the missing position is this reference word. The larger the predicted probability value, the higher the likelihood that the word missing at the missing position is this reference word. On this basis, the reference words ranked in the top preset order can be selected in descending order of the predicted probability value as the candidate words for the missing position. The setting method of the preset order can refer to the relevant description above and will not be elaborated here. By the above method, obtaining the predicted probability values of each reference word by using the individual semantic representation and the word semantic representations of each reference word, and then selecting the reference words ranked in the top preset order in descending order of the predicted probability value as the candidate words for the missing position, can thus select reference words as candidate words for the missing position based on the individual semantic representation and the word semantic representation, which is conducive to improving the accuracy of the candidate words.
[0164] In a specific implementation scenario, as described above, the text to be processed is sent to the second prediction network to obtain the individual semantic representation of the missing position. For the convenience of description, the individual semantic representation can be recorded as h, and the word semantic representation of each reference word can be recorded as W. It should be noted that W is the set of word semantic representations of each reference word, and the probability prediction value p can be calculated by the following formula:
[0165] p=softmax(h·W)……(6)
[0166] In the above formula (6), p represents the predicted probability value of each reference word, h represents the individual semantic representation of the missing position, W represents the word semantic representation of each reference word, · represents the dot product operation, and softmax represents the normalization process.
[0167] In another specific implementation scenario, please refer to Figure 8 , Figure 8 FIG. 1 is a schematic diagram of a state of an embodiment of a reference word prediction process. Figure 8 As shown in the figure, after the to-be-completed text "World Intellectual Property Headquarters is located in ()" is processed into the above-mentioned to-be-processed text, it is sent to the second prediction network to obtain the individual semantic representation h of the missing position, and the m reference words (W1, W2, W3, ..., W m ) After vectorization, we can get m corresponding semantic representations of words (v1, v2, v3, …, v m ), thereby combining the individual semantic representation h with the m word semantic representations (v1, v2, v3, …, v m ) to perform dot product operations to obtain the predicted probability values of each reference word (p1, p2, p3, ..., p m ), and sort the reference words in descending order of predicted probability values, and finally select the reference words with the first preset order (such as the first two), such as "Geneva" and "New York" as candidate words for the missing position. It should be noted that Figure 8 The example shown is only one possible situation in the actual application process, and does not limit other possible situations in the actual application process. The specific settings can be made according to the actual application situation and are not limited here.
[0168] It should be noted that when the text to be completed includes multiple missing positions, the above method can be used to perform completion prediction on each missing position, and finally obtain candidate words for each missing position. The specific process can be referred to the above description and will not be repeated here.
[0169] Different from the above-mentioned embodiment, by performing semantic extraction on the text to be completed, the individual semantic representation of each missing position is obtained, so that for each missing position, the individual semantic representation of the missing position and the semantic representation of each reference word are used to directly obtain at least one candidate word for the missing position, which can help improve the accuracy and efficiency of the completion prediction. In addition, since the candidate words are directly predicted for the missing position, the missing position is not limited to missing text, words or entities, which can help to achieve the prediction of mixed granularity such as text, words, and entities.
[0170] See also Fig. 9 , Fig. 9 1 is a flow chart of an embodiment of a second prediction network training process. Specifically, it may include the following steps:
[0171] Step S91: perform semantic extraction on the sample text using the second prediction network to obtain individual semantic representations of the samples at the missing positions of each sample.
[0172] In the embodiment of the present disclosure, the sample text includes at least one sample missing position. The process of obtaining the sample text can be specifically referred to in the aforementioned embodiment and the appendix of the specification. Figure 3 The relevant description in will not be repeated here.
[0173] The specific method of semantic extraction of the sample text can be found in the description of step S71 in the aforementioned disclosed embodiment, which will not be repeated here.
[0174] Step S92: For each sample missing position, the sample individual semantic representation of the sample missing position and the word semantic representation of each reference word are used to obtain the sample prediction probability value of each reference word.
[0175] For details, please refer to the relevant description of step S72 in the aforementioned disclosed embodiment, which will not be repeated here.
[0176] Step S93: Obtain a second loss value of the second prediction network based on the sample prediction probability values of each reference word at each sample missing position.
[0177] Specifically, the second loss value can be calculated using the cross entropy loss function, which can be specifically expressed as:
[0178]
[0179] In the above formula (7), M represents the number of missing positions in the sample text, y i is the default word corresponding to the i-th missing position in the sample text, p i Represents the sample prediction probability value of each reference word predicted at the i-th missing position in the sample text.
[0180] Step S94: using the second loss value to adjust the network parameters of the second prediction network.
[0181] Specifically, the network parameters of the second prediction network can be adjusted using the second loss value by using methods such as Stochastic Gradient Descent (SGD), Batch Gradient Descent (BGD), and Mini-Batch Gradient Descent (MBGD). Batch gradient descent refers to using all samples to update parameters at each iteration; Stochastic gradient descent refers to using one sample to update parameters at each iteration; Mini-batch gradient descent refers to using a batch of samples to update parameters at each iteration, which will not be repeated here.
[0182] Different from the above-mentioned embodiment, by using the second prediction network to perform semantic extraction on the sample text, the individual semantic representation of the sample at each missing position of the sample is obtained, so that for each missing position of the sample, the individual semantic representation of the sample at the missing position of the sample and the semantic representation of the word of each reference word are used to obtain the sample prediction probability value of each reference word, and then based on the sample prediction probability value of each reference word at each missing position of the sample, the second loss value of the second prediction network is obtained, and on this basis, the network parameters of the second prediction network are adjusted using the second loss value. Therefore, the word semantic representation of the reference word is used to assist in word prediction at the missing position of the sample, and the network parameters of the second prediction network are adjusted based on the statistically obtained second loss value, which can help improve the accuracy of the second prediction network. In addition, since the overall direct prediction of the missing position of the sample is performed, the missing position of the sample is not limited to missing text, words or entities, which can help to achieve prediction of mixed granularities such as text, words, and entities.
[0183] See also Fig.10 , Fig.10 yes Figure 1Flow chart of another embodiment of step S13 in the figure. Specifically, the disclosed embodiment is a flow chart of an embodiment of performing a complement prediction on the text to be completed when the source situation includes a second text library related to a preset knowledge field. As described in the aforementioned disclosed embodiment, the knowledge graph and text library corresponding to the preset knowledge field can be used to perform a complement prediction on the text to be completed. In the disclosed embodiment, the knowledge graph may include a number of triples, and the triples may include two entities and an entity relationship between the two entities. The triples may be specifically expressed as <entity 1, entity relationship, entity 2>. Taking the preset knowledge field of classical music as an example, a number of triples may include but are not limited to: <Mozart, birthplace, Salzburg>, <Mozart, school, classical school>, <Austria, the oldest history, Salzburg>, <Salzburg, long-standing anniversary, Mozart Week>, and when the preset knowledge field is other situations, the same can be applied, and examples are not given one by one here. The disclosed embodiment may specifically include the following steps:
[0184] Step S1010: Perform entity search in a number of triples to obtain a target triple that contains a target entity.
[0185] In the disclosed embodiment, the target entity is an entity extracted from the text to be completed. Specifically, a natural language processing (NLP) tool (such as LTP, etc.) can be used to perform named entity recognition on the text to be completed, so that the target entity can be extracted from the text to be completed. Taking the aforementioned text to be completed "As one of the representatives of the Vienna Classical School, () was appointed as the court musician of Salzburg in 1772" as an example, the target entity "Salzburg" can be extracted therefrom, and other situations can be deduced by analogy, and no examples are given one by one here.
[0186] In an implementation scenario, entity search can be performed in the knowledge graph corresponding to the preset knowledge field, and the triple containing the target entity can be directly used as the target triple. Still taking the text to be completed "As one of the representatives of the Vienna Classical School, () was appointed as the court musician of Salzburg in 1772" as an example, the triple containing the target entity "Salzburg": <Mozart, birthplace, Salzburg>, <Austria, the oldest history, Salzburg>, <Salzburg, anniversary, Mozart Week> can be directly used as the target triple. When the text to be completed is other cases, it can be deduced by analogy, and examples are not given one by one here. The above method directly uses the triple containing the target entity as the target triple, which can increase the speed of searching for the target triple.
[0187] In another implementation scenario, the triple containing the target entity can be used as a candidate triple, and another entity other than the target entity in the candidate triple can be used as a reference entity. On this basis, the word semantic representation of each reference word in the second text library is used to obtain the entity semantic representation of the reference entity, and the overall semantic representation of the text to be completed is obtained, so as to select at least one candidate triple as the target triple based on the similarity between the entity semantic representation of each reference entity and the overall semantic representation. The above method, by taking the triple containing the target entity as the candidate triple, and taking another entity other than the target entity in the candidate triple as the reference entity, thereby utilizing the word semantic representation of each reference word in the text library to obtain the overall semantic representation of the text to be completed and the entity semantic representation of the reference entity, and then based on the similarity between the entity semantic representation of each reference entity and the overall semantic representation, select at least one candidate triple as the target triple. Therefore, the candidate triples can be further screened based on the similarity, which can help reduce the interference of triples with lower similarity on subsequent completion predictions, and help reduce the complexity of subsequently integrating the target triple into the text to be completed.
[0188] In a specific implementation scenario, the overall semantic representation can be obtained by fusing the semantic representations of each word in the text to be completed. Specifically, the semantic representations of each word in the text to be completed can be expressed as:
[0189]
[0190] In the above formula (8), V seq represents the overall semantic representation of the text to be completed, n represents the total number of words in the text to be completed, V ti Indicates the tth i The semantic representation of each word. Take the text to be completed as "As one of the representatives of the Vienna Classical School, () was appointed as the court musician of Salzburg in 1772" as an example. The words in the text to be completed include: as, Vienna, classical, school, representative, person, one of, 1772, year, appointed, Salzburg, court, musician. The semantic representation of the above words can be substituted into the above formula (8) to obtain the overall semantic representation of the text to be completed. Other cases can be deduced by analogy, and no examples are given here one by one.
[0191] In another specific implementation scenario, the similarity between the entity semantic representation and the overall semantic representation may be cosine similarity, and the similarity S may be obtained by the following formula:
[0192] S=cosine(V seq ,V ent2 )……(9)
[0193] In the above formula (9), V seq Represents the overall semantic representation of the text to be completed, The entity semantic representation of another entity (i.e., the reference entity) other than the target entity in the target triple. Still taking the text to be completed "As one of the representatives of the Vienna Classical School, () was appointed as the court musician of Salzburg in 1772" as an example, while using the above formula (8) to obtain the overall semantic representation of the text to be completed, the candidate triples can be obtained respectively: <Mozart, birthplace, Salzburg>, <Austria, the oldest history, Salzburg>, <Salzburg, anniversary, Mozart Week>, the entity semantic representation of the reference entity "Mozart" other than the target entity "Salzburg", the entity semantic representation of the reference entity "Austria", the entity semantic representation of the reference entity "Mozart Week", and the entity semantic representation of the reference entity "Mozart Week", and based on the above formula (9), the similarity between the overall semantic representation and the above three entity semantic representations is obtained. Other cases can be deduced by analogy, and no examples are given here one by one.
[0194] In another specific implementation scenario, the candidate triples can be sorted in order from large to small according to the similarity, so as to select the candidate triples in the front preset order as the target triples. Specifically, the preset order can be set according to the actual application needs, for example, it can be set to 2, 3, 4, etc. In addition, in order to reduce the interference of the candidate triples with lower similarity on the subsequent completion prediction, and reduce the complexity of integrating the target triples into the text to be completed, and avoid the low completion prediction accuracy due to too few target triples, the preset order can be specifically set to 2, that is, the candidate triples in the first two positions after the similarity is sorted from large to small can be used as the target triples. Still taking the text to be completed "as one of the representatives of the Vienna Classical School, () was appointed as the Salzburg court musician in 1772" as an example, <Mozart, birthplace, Salzburg>, <Salzburg, anniversary, Mozart week> can be selected as the target triples. Other situations can be deduced by analogy, and no more examples are given here.
[0195] In another specific implementation scenario, the second text library may include at least one reference text, and the reference text includes at least one reference word, so that at least one reference text can be segmented and tagged with parts of speech to obtain a number of words tagged with part-of-speech categories, and the words with the part-of-speech category as a preset category can be segmented word by word, and the segmented words and unsegmented words can be used to obtain a number of reference words, and then semantic extraction is performed on the reference words to obtain word semantic representations of the reference words. For details, please refer to the relevant description in the aforementioned disclosed embodiment, which will not be repeated here.
[0196] Step S1020: Integrate the target triple into the target entity of the text to be completed to obtain a fused text.
[0197] In an implementation scenario, after obtaining the target triplet, the reference entity, the target entity, and the entity relationship between the two in the target triplet can be extracted, and the reference entity and entity relationship therein can be inserted to the left and / or right side of the target entity of the text to be completed, thereby obtaining a fused text. Still taking the text to be completed "As one of the representatives of the Vienna Classical School, () was appointed as the court musician of Salzburg in 1772" as an example, the reference entity (i.e. Mozart), the target entity (i.e. Salzburg) and the entity relationship (i.e. birthplace) can be extracted from the target triplet <Mozart, birthplace, Salzburg>, and the reference entity (i.e. Mozart Week), the target entity (i.e. Salzburg) and the entity relationship (anniversary) can be extracted from another target triplet <Salzburg, anniversary, Mozart Week>, and the reference entity "Mozart", the entity relationship "birthplace" and the reference entity "Mozart Week", the entity relationship "anniversary" can be inserted to the left side of the target entity "Salzburg" of the text to be completed, thereby obtaining the fused text "As one of the representatives of the Vienna Classical School, () was appointed as the court musician of Salzburg in 1772", thereby obtaining the fused text "As one of the representatives of the Vienna Classical School, Mozart ... As one of the representatives of the Vienna Classical School, () was appointed as the court musician of Salzburg, the birthplace of Mozart, on the Mozart Week in 1772", or, inserted to the right side of the target entity "Salzburg" of the full text to be completed, the fused text is obtained as "As one of the representatives of the Vienna Classical School, () was appointed as the court musician of Salzburg, the birthplace of Mozart, on the Mozart Week in 1772", or, inserted to the left and right sides of the target entity "Salzburg" of the full text to be completed, the fused text is obtained as "As one of the representatives of the Vienna Classical School, () was appointed as the court musician of Salzburg, the birthplace of Mozart, on the Mozart Week in 1772", without limitation here. Other situations can be deduced by analogy, and no examples are given here one by one.
[0198] In another implementation scenario, after obtaining the target triple, the target triple can also be used to construct a knowledge tree, and the knowledge tree is converted into a text sequence, and the root node of the knowledge tree is the target entity, the leaf node of the knowledge tree is the reference entity, the reference entity is another entity other than the target entity in the target triple, and the intermediate node between the root node and the leaf node is the entity relationship between the target entity and the reference entity, on this basis, the text sequence can be integrated into the target entity of the text to be completed to obtain a fused text. The above method uses the target triple to construct a knowledge tree, and converts the knowledge tree into a text sequence, thereby integrating the text sequence into the target entity of the text to be completed to obtain a fused text, so it can be beneficial to convert the target triple into a text sequence with structured features by constructing a knowledge tree, which can further improve the readability of the fused text and help improve the accuracy of subsequent completion predictions.
[0199] In a specific implementation scenario, please refer to Fig.11 , Fig.11 It is a framework diagram of an embodiment of a knowledge tree. As shown in Figure 11, still taking the text to be completed "As one of the representatives of the Vienna Classical School, () was appointed as the court musician of Salzburg in 1772" as an example, as mentioned above, its corresponding target triples include: <Mozart, birthplace, Salzburg>, <Salzburg, anniversary, Mozart Week>, so the target entity "Salzburg" can be used as the root node of the knowledge tree, and the reference entities "Mozart" and "Mozart Week" can be used as leaf nodes, and the entity relationships "birthplace" and "anniversary" can be used as intermediate nodes between the root node and the leaf node respectively. It should be noted that in the disclosed embodiment and the following disclosed embodiments, unless otherwise specified, the root node represents a node in the knowledge tree where there is no parent node, and the leaf node represents a node in the knowledge tree where there is no child node.
[0200] In another specific implementation scenario, the above-mentioned knowledge tree is a binary tree. On this basis, the in-order traversal method can be used to traverse the knowledge tree in order, and the combination of words traversed in order is used as a text sequence. In the disclosed embodiment and the following disclosed embodiments, the in-order traversal method is a type of binary tree traversal, which can also be called middle root traversal or in-order traversal. When the in-order traversal method is used, the left subtree is traversed first, then the root node is visited, and finally the right subtree is traversed. Please refer to Fig.11 ,by Fig.11 Taking the knowledge tree shown in the figure as an example, when the knowledge tree is traversed in the middle order traversal method, the left subtree, namely, "Mozart", "birthplace", is traversed first, then the root node, namely, "Salzburg", is visited, and finally the right subtree, namely, "anniversary", "Mozart week", is traversed. On this basis, the combination of the words traversed in order, namely, "Mozart birthplace Salzburg anniversary Mozart week", is used as a text sequence. Other situations can be deduced by analogy, and no examples are given here one by one.
[0201] In another specific implementation scenario, after obtaining the text sequence, the target entity in the to-be-completed text can be replaced with the text sequence to obtain a fused text. Fig.12 , Fig.12 FIG. 1 is a state diagram of an embodiment of a fusion text acquisition process. Fig.12As shown, still taking the text to be completed "As one of the representatives of the Vienna Classical School, () was appointed as the court musician of Salzburg in 1772" as an example, after the above entity search, knowledge tree construction and conversion, the text sequence "Mozart's birthplace Salzburg anniversary Mozart Week" can be obtained. On this basis, the target entity in the text to be completed can be directly replaced with the text sequence to obtain the fused text "As one of the representatives of the Vienna Classical School, () was appointed as the court musician of Mozart's birthplace Salzburg anniversary Mozart Week in 1772". Other situations can be deduced by analogy, and no examples are given here one by one.
[0202] Step S1030: Use the second text library to perform completion prediction on the fused text to obtain at least one candidate word for the missing position.
[0203] Specifically, the second text library includes at least one reference text, and the reference text contains at least one reference word. The semantic extraction of the reference word can obtain the semantic representation of the reference word. The specific process can refer to the relevant description in the aforementioned public embodiment, which will not be repeated here. On this basis, the words belonging to the to-be-completed text in the fused text can be encoded with a first digital sequence in order of position, and the words belonging to the target triples in the fused text can be encoded with a second digital sequence in order, and the largest first digital sequence is smaller than the smallest second digital sequence, so that the encoded fused text is semantically extracted to obtain the individual semantic representation of each missing position, and then for each missing position, the individual semantic representation of the missing position and the semantic representation of each reference word can be used to obtain at least one candidate word for the missing position. The above method, by encoding the words in the fused text belonging to the text to be completed in the order of position with the first digital order, and encoding the words in the fused text belonging to the target triplet with the second digital order, and the largest first digital order is smaller than the smallest second digital order, can integrate domain knowledge while keeping the original word order of the text to be completed unchanged in the completion prediction process. On this basis, a second semantic extraction is performed on the encoded fused text to obtain the individual semantic representation of each missing position, and the individual semantic representation of the missing position and the word semantic representation of each reference word are used to obtain at least one candidate word for the missing position, which can help improve the accuracy of the individual semantic representation, and further can help improve the accuracy of the completion prediction.
[0204] In an implementation scenario, as described in the aforementioned disclosed embodiment, before encoding in the order of position, a default symbol can be added to each missing position of the fused text. The specific setting method of the default symbol can refer to the relevant description in the aforementioned disclosed embodiment, for example, it can be set to [mask], which is not limited here.
[0205] In another implementation scenario, [CLS] and [SEP] can be added at the beginning and end of the fused text as the start and end marks, respectively. Taking the above fused text "As one of the representatives of the Vienna Classical School, () was appointed as the court musician of Mozart Week in Salzburg, the birthplace of Mozart, in 1772" as an example, it can be processed as "[CLS] As one of the representatives of the Vienna Classical School, [mask] was appointed as the court musician of Mozart Week in Salzburg, the birthplace of Mozart, in 1772 [SEP]", and other situations can be deduced by analogy, and no examples are given here one by one.
[0206] In another implementation scenario, in order to facilitate the prediction of mixed granularities such as text, words, and entities, the fused text can be further segmented and tagged with parts of speech to obtain a number of words marked with part-of-speech categories, and the words with part-of-speech categories as preset categories can be segmented word by word. The specific methods of word segmentation and part-of-speech tagging can refer to the relevant description in the aforementioned public embodiment, which will not be repeated here. In addition, the preset category can be set to a place name, which can be specifically referred to the relevant description in the aforementioned public embodiment, which will not be repeated here. Still taking the fused text "As one of the representatives of the Vienna Classical School, () was appointed as the court musician of Mozart Week in Salzburg, Mozart's birthplace, in 1772" as an example, the relevant steps in the aforementioned public embodiment can be used to fill in the default symbols, start marks and end marks, as well as word segmentation, part-of-speech tagging and word-by-word segmentation, and finally the fused text can be processed as follows:
[0207]
[0208] When the fused text is other texts, the same logic can be applied and examples will not be given here one by one.
[0209] In another implementation scenario, in order to distinguish between words belonging to the text to be completed and words belonging to the target triple, a sequence start mark can be added before the text sequence in the fused text, and a sequence end mark can be added after the text sequence. The sequence start mark and the sequence end mark can be set according to actual application needs. For example, <s>As a sequence start mark, <t>As a sequence end mark. On this basis, the fusion text "As one of the representatives of the Vienna Classical School, () was appointed as the court musician of Mozart Week in Salzburg, Mozart's birthplace, in 1772" can be processed as:
[0210]
[0211] In another implementation scenario, since the target entity exists in both the target triple and the text to be completed, in order to further maintain the original word order of the text to be completed, the target entity can be regarded as a word belonging to the text to be completed, that is, the target entity is encoded as the first digital sequence. In addition, the second digital sequence can be encoded immediately after the first digital digit. For example, if the largest first digital sequence is i, the smallest second digital sequence can be i+1. Still taking the above-mentioned text to be completed "As one of the representatives of the Vienna Classical School, () was appointed as the Salzburg court musician in 1772" as an example, the fused text after position encoding can be expressed as:
[0212]
[0213] In addition, as described in the aforementioned disclosed embodiments, when the source situation includes a second text library derived from a preset knowledge field, the completion prediction can be performed using a third prediction network, and the third prediction network can be specifically obtained by training a preset neural network (such as BERT) using sample text. On this basis, the encoded fused text can be sent to the third prediction network, so that the individual semantic representation of each missing position can be obtained. The process of obtaining the sample text can be specifically referred to the relevant description in the aforementioned disclosed embodiments, which will not be repeated here. In addition, the training process of the third prediction network can be specifically referred to the following disclosed embodiments, which will not be repeated here.
[0214] In an implementation scenario, the individual semantic representation and the semantic representation of each reference word can obtain the predicted probability value of each reference word. Specifically, the predicted probability value of the reference word can indicate the possibility that the word missing at the missing position is the reference word. The larger the predicted probability value, the higher the possibility that the word missing at the missing position is the reference word. On this basis, the reference words at the front preset positions can be selected in descending order of predicted probability values as candidate words for the missing position. The setting method of the preset positions can refer to the aforementioned related description, which will not be repeated here.
[0215] In a specific implementation scenario, as described above, the encoded fused text is sent to the third prediction network to obtain the individual semantic representation of the missing position. For the convenience of description, the individual semantic representation can be recorded as h, and the word semantic representation of each reference word can be recorded as W. It should be noted that W is the set of word semantic representations of each reference word, and the probability prediction value p can be calculated by the following formula:
[0216] p=softmax(h·W)……(10)
[0217] In the above formula (10), p represents the predicted probability value of each reference word, h represents the individual semantic representation of the missing position, W represents the word semantic representation of each reference word, · represents the dot product operation, and softmax represents the normalization process. For details, please refer to the relevant description in the above disclosed embodiment, which will not be repeated here.
[0218] It should be noted that, when the target triple containing the target entity cannot be searched, the fused text is the text to be completed itself. In this case, the completion prediction cannot refer to the domain knowledge, that is, the text to be completed can be directly semantically extracted to obtain the individual semantic representation of each missing position, and for each missing position, the individual semantic representation of the missing position and the word semantic representation of each reference word are used to obtain at least one candidate word for the missing position. For details, please refer to the relevant description in the aforementioned public embodiment, which will not be repeated here. Therefore, regardless of whether the target triple can be searched, the text to be completed can be predicted for completion, so that the domain knowledge can be used in a pluggable manner, which can greatly improve the flexibility of the completion prediction.
[0219] In addition, when the knowledge graph is updated, the searched target triples may also change. In this case, the steps in the disclosed embodiment can still be used to predict at least one candidate word for the missing position. Therefore, whether the knowledge graph is updated or not will not affect the subsequent completion prediction, which can greatly improve the scalability of the completion prediction.
[0220] In addition, when the text to be completed includes multiple missing positions, the above method can be used to perform completion prediction on each missing position, and finally obtain candidate words for each missing position. The specific process can be referred to the above description and will not be repeated here.
[0221] Different from the above-mentioned embodiment, by performing entity search in several triples, a target triple containing a target entity is obtained, and the target triple is integrated into the target entity of the text to be completed to obtain a fused text, so as to use the second text library to perform completion prediction on the fused text to obtain at least one candidate word for the missing position. Therefore, by searching to obtain a target triple containing a target entity, and integrating the target triple into the target entity of the text to be completed, the field knowledge closely related to the text to be completed can be integrated into the text to be completed, which can be conducive to further improving the accuracy of subsequent completion prediction.
[0222] See also Fig.13 , Fig.13 1 is a flow chart of an embodiment of the third prediction network training process. In the disclosed embodiment, the sample knowledge graph includes a number of sample triples, and the sample triples include two sample entities and a sample entity relationship between the two sample entities. For details, please refer to the relevant description in the aforementioned disclosed embodiment, which will not be repeated here. The disclosed embodiment may specifically include the following steps:
[0223] Step S1310: Perform entity search in a number of sample triples to obtain a sample target triple that contains a sample target entity.
[0224] In the embodiment of the present disclosure, the sample target entity is an entity extracted from the sample text. For details, please refer to the relevant description of step S1010 in the above-mentioned embodiment of the disclosure, which will not be repeated here.
[0225] Step S1320: Integrate the sample target triple into the sample target entity of the sample text to obtain the sample fused text.
[0226] For details, please refer to the relevant description of step S1020 in the aforementioned disclosed embodiment, which will not be repeated here.
[0227] Step S1330: According to the order of position, the words in the sample fusion text belonging to the sample text are encoded with a first sample digital sequence, and the words in the sample fusion text belonging to the sample target triplet are encoded with a second sample digital sequence.
[0228] In the embodiment of the present disclosure, the largest first sample digital sequence is smaller than the smallest second sample digital sequence. For details, please refer to the relevant description of step S1030 in the aforementioned embodiment of the present disclosure, which will not be repeated here.
[0229] Step S1340: using the third prediction network to perform semantic extraction on the encoded sample fusion text to obtain individual semantic representations of samples at each sample missing position.
[0230] For details, please refer to the relevant description of step S1030 in the aforementioned disclosed embodiment, which will not be repeated here.
[0231] Step S1350: For each sample missing position, the sample individual semantic representation of the sample missing position and the word semantic representation of each reference word are used to obtain the sample prediction probability value of each reference word.
[0232] For details, please refer to the relevant description of step S1030 in the aforementioned disclosed embodiment, which will not be repeated here.
[0233] Step S1360: Obtain a third loss value of the third prediction network based on the sample prediction probability values of each reference word at each sample missing position.
[0234] Specifically, the third loss value can be calculated using the cross entropy loss function, which can be specifically expressed as:
[0235]
[0236] In the above formula (7), M represents the number of missing positions in the sample text, y i is the default word corresponding to the i-th missing position in the sample text, p i Represents the sample prediction probability value of each reference word predicted at the i-th missing position in the sample text.
[0237] Step S1370: using the third loss value, adjusting the network parameters of the third prediction network.
[0238] Specifically, the network parameters of the third prediction network can be adjusted using the third loss value by using methods such as Stochastic Gradient Descent (SGD), Batch Gradient Descent (BGD), and Mini-Batch Gradient Descent (MBGD). Batch gradient descent refers to using all samples to update parameters at each iteration; Stochastic gradient descent refers to using one sample to update parameters at each iteration; Mini-batch gradient descent refers to using a batch of samples to update parameters at each iteration, which will not be repeated here.
[0239] Different from the above-mentioned embodiment, a sample target triplet containing a sample target entity is obtained by performing entity search in a plurality of sample triples, and the sample target triplet is integrated into the sample target entity of the sample text to obtain a sample fusion text, and further, according to the order of position, the words belonging to the sample text in the sample fusion text are encoded with a first sample digital sequence, and the words belonging to the sample target triplet in the sample fusion text are encoded with a second sample digital sequence, so that the encoded sample fusion text is semantically extracted by using a third prediction network to obtain the sample individual semantic representation of each sample missing position, and for each sample missing position, The sample individual semantic representation of the sample missing position and the word semantic representation of each reference word are used to obtain the sample prediction probability value of each reference word, and then based on the sample prediction probability value of each reference word at each sample missing position, the third loss value of the third prediction network is obtained, and the network parameters of the third prediction network are adjusted using the third loss value. Therefore, the sample target triplet containing the sample target entity is obtained by searching, and the sample target triplet is integrated into the sample target entity of the sample text, so that the domain knowledge closely related to the sample text can be integrated into the sample text, which can help to further improve the accuracy of the third prediction network.
[0240] It should be noted that the completion prediction method matching different source situations in this application can be as follows: Figure 2 The various disclosed embodiments shown and described above are integrated into the same system framework, or they can be implemented separately and independently.
[0241] See also Fig.14 , Fig.14 This is a flowchart of another embodiment of the text completion method of the present application. Specifically, it may include the following steps:
[0242] Step S1410: Acquire the text to be completed, and determine that the source of the missing content of the text to be completed is unknown.
[0243] In the embodiment of the present disclosure, the text to be completed includes at least one missing position. For details, please refer to the relevant steps in the above-mentioned embodiment of the disclosure, which will not be repeated here.
[0244] Step S1420: Predict the text to be completed word by word to obtain at least one candidate word for the missing position.
[0245] In one implementation scenario, a preset number of default symbols can be added to each missing position of the text to be completed to obtain the text to be processed, and for each missing position, the text to be processed is predicted several times to obtain the predicted text of the default symbol at the corresponding order of the prediction times, and the candidate words of the missing position are obtained based on the predicted texts predicted several times. For details, please refer to the relevant description in the aforementioned disclosed embodiment, which will not be repeated here.
[0246] In another implementation scenario, different from the above description, for the i-th missing position, the candidate words predicted at the missing position can be filled in at the 1st to i-1th missing positions, and a preset number of default symbols can be filled in at the i-th to N-th missing positions, so that several texts to be processed can be obtained. For each text to be processed, several predictions can be made, and the predicted text of the default symbol at the corresponding order of the prediction times can be obtained, and the combination of the predicted texts predicted several times can be used as the candidate words corresponding to the text to be processed, and then the candidate words corresponding to each text to be processed can be used as the candidate words for the i-th missing position, and so on, until the candidate words for all N missing positions are obtained. In the above method, the prediction of the candidate word at the next missing position depends on the candidate word at the previous missing position, which can help improve the correlation between the candidate words at each missing position in the completion prediction process, and can help gradually improve the accuracy of the candidate words at each missing position in the completion prediction process for each missing position.
[0247] In a specific implementation scenario, taking the text to be completed "() Medical journal The Lancet published online the results of the Phase II clinical trial of XX vaccine of the () Military Medical Research Institute" as an example, the text to be completed includes 2 (that is, N is 2) missing positions. For the first missing position, a preset number of default symbols can be filled in the two missing positions respectively as the text to be processed, and the text to be processed is used to perform several predictions, and finally a combination of the predicted texts predicted several times is used as the candidate words corresponding to the text to be processed (such as "Britain" and "United States"). The specific process of predicting text can refer to the relevant description in the aforementioned public embodiment, which will not be repeated here. On this basis, for the second missing position, the candidate words predicted at the first missing position can be added to the first missing position, that is, the above candidate words "UK" and "USA" are added to the first missing position, and a preset number of default symbols are added to the second missing position to obtain two texts to be processed. Several predictions can be made for these two texts to be processed. Finally, for the text to be processed with "UK" added to the first missing position, the candidate word "China" can be predicted at the second missing position, and for the text to be processed with "USA" added to the first missing position, the candidate word "Japan" can be predicted at the second missing position. At this point, all two missing positions have been completed and predicted. Finally, the candidate words for the first missing position include "UK" and "USA", and the candidate words for the second missing position include "China" and "Japan". The above example is only one possible situation in the actual application process, and does not limit other possible situations. In addition, in the case of other numbers of missing positions, it can be deduced by analogy, and no examples are given here one by one.
[0248] Step S1430: using the candidate words at each missing position to obtain the complete text of the text to be completed.
[0249] In an implementation scenario, a corresponding candidate word can be added to each missing position, so that several candidate texts of the text to be completed can be obtained, so that the final score of each candidate text can be obtained, and based on the final scores of the several candidate texts, a candidate text is selected as the complete text of the text to be completed. For details, please refer to the relevant description in the aforementioned disclosed embodiment, which will not be repeated here.
[0250] In another implementation scenario, in order to improve the accuracy of the final score, a corresponding candidate word can be added to each missing position to obtain several candidate texts of the text to be completed, and for each candidate text, the words in the candidate text are sorted in reverse order to obtain the reverse text of the candidate text, thereby obtaining the final score of the candidate text based on the first score of the candidate text and the second score of the reverse text, and then based on the final scores of the several candidate texts, a candidate text is selected as the complete text of the text to be completed. For details, please refer to the relevant description in the aforementioned disclosed embodiment, which will not be repeated here.
[0251] Different from the above-mentioned embodiment, by obtaining the text to be completed and determining that the source of the missing content of the text to be completed is unknown, the text to be completed is predicted word by word, and at least one candidate word of the missing position is obtained, and then the candidate words of each missing position are used to obtain the complete text of the text to be completed. Therefore, the missing content of the text to be completed can be completed without manual labor, which can improve the efficiency of text completion and reduce the cost of text completion. In addition, when the source of the missing content is unknown, word-by-word prediction can help improve the accuracy of text completion.
[0252] See also Fig.15 , Fig.15 This is a flowchart of another embodiment of the text completion method of the present application. Specifically, it may include the following steps:
[0253] Step S1510: Acquire the text to be completed, and determine the text library from which the missing content of the text to be completed originates.
[0254] In the embodiment of the present disclosure, the text to be completed includes at least one missing position. For details, please refer to the relevant steps in the above-mentioned embodiment of the disclosure, which will not be repeated here.
[0255] Step S1520: Use the text library to perform completion prediction on the text to be completed, and obtain at least one candidate word for the missing position.
[0256] In an implementation scenario, the text library includes at least one reference text, and the reference text includes at least one reference word. On this basis, semantic extraction can be performed on the text to be completed to obtain individual semantic representations of each missing position, and for each missing position, the individual semantic representation of the missing position and the semantic representation of each reference word are used to obtain at least one candidate word for the missing position. For details, please refer to the relevant description in the aforementioned disclosed embodiment, which will not be repeated here.
[0257] In another implementation scenario, as mentioned above, the text library includes at least one reference text, and the reference text includes at least one reference word, so that the reference word can be semantically extracted to obtain the word semantic representation of the reference word. The specific process can refer to the relevant description in the aforementioned public embodiment, which will not be repeated here. On this basis, the text to be completed can be segmented to obtain several words, and the word semantic representation of the reference word that is consistent with the word in the text to be completed can be used as the word semantic representation of the word, and then the word semantic representation of several words in the text to be completed can be merged to obtain the overall semantic representation of the text to be completed. For example, the word semantic representation is a vector containing elements of a preset dimension (such as 128 dimensions), then the elements in the same position of the word semantic representations of several words can be averaged to obtain the overall semantic representation of the text to be completed. Furthermore, the similarity between the word semantic representation and the overall semantic representation of each reference word in the text library can be obtained respectively (e.g., cosine similarity), so that the reference words can be sorted in order of similarity from large to small, and then the reference words located in the front preset order (e.g., the top 5) can be selected as candidate words for each missing word position in the text to be completed.
[0258] Step S1530: using the candidate words at each missing position to obtain the complete text of the text to be completed.
[0259] In an implementation scenario, a corresponding candidate word can be added to each missing position, so that several candidate texts of the text to be completed can be obtained, so that the final score of each candidate text can be obtained, and based on the final scores of the several candidate texts, a candidate text is selected as the complete text of the text to be completed. For details, please refer to the relevant description in the aforementioned disclosed embodiment, which will not be repeated here.
[0260] In another implementation scenario, in order to improve the accuracy of the final score, a corresponding candidate word can be added to each missing position to obtain several candidate texts of the text to be completed, and for each candidate text, the words in the candidate text are sorted in reverse order to obtain the reverse text of the candidate text, thereby obtaining the final score of the candidate text based on the first score of the candidate text and the second score of the reverse text, and then based on the final scores of the several candidate texts, a candidate text is selected as the complete text of the text to be completed. For details, please refer to the relevant description in the aforementioned disclosed embodiment, which will not be repeated here.
[0261] Different from the aforementioned embodiment, by obtaining the text to be completed and determining the text library from which the missing content of the text to be completed originates, the text library is used to perform completion prediction on the text to be completed, and at least one candidate word for the missing position is obtained, and then the candidate words for each missing position are used to obtain the complete text of the text to be completed. Therefore, the missing content of the text to be completed can be completed without relying on manual labor, which can improve the efficiency of text completion and reduce the cost of text completion. In addition, since the missing content is determined to originate from the text library, the text library is used to perform completion prediction on the text to be completed, and at least one candidate word for the missing position is directly obtained, that is, the missing unknown is not limited to missing text, words or entities, which can be conducive to the prediction of mixed granularity such as text, words, and entities.
[0262] See also Fig.16 , Fig.16 This is a flowchart of another embodiment of the text completion method of the present application. Specifically, it may include the following steps:
[0263] Step S1610: Acquire the text to be completed, and determine the text library from which the missing content of the text to be completed originates.
[0264] In the embodiment of the present disclosure, the text to be completed includes at least one missing position, and the text library involves a preset knowledge field. For details, please refer to the relevant steps in the above-mentioned embodiment of the disclosure, which will not be repeated here.
[0265] Step S1620: Utilize the knowledge graph and text library corresponding to the preset knowledge domain to perform completion prediction on the text to be completed, and obtain at least one candidate word for the missing position.
[0266] In an implementation scenario, the knowledge graph includes several triples, and the triples include two entities and the entity relationship between the two entities. On this basis, entity search can be performed in several triples to obtain a target triple containing a target entity, and the target entity is an entity extracted from the text to be completed, and the target triple is integrated into the target entity of the text to be completed to obtain a fused text, so as to use the text library to perform completion prediction on the fused text and obtain at least one candidate word for the missing position. For details, please refer to the relevant description in the aforementioned disclosed embodiment, which will not be repeated here.
[0267] In another implementation scenario, similar to the above description, the knowledge graph includes several triples, and the triples include two entities and the entity relationship between the two entities. On this basis, entity search can be performed in several triples to obtain a target triple containing a target entity, and the target entity is an entity extracted from the text to be completed. Different from the above description, while searching for the target triple, the text library can be directly used to complete the text to be completed, and at least one candidate word for the missing position can be obtained. The specific process can refer to the relevant description in the above disclosed embodiment, which will not be repeated here. On this basis, another entity other than the target entity in the target triple can be extracted, as described in the above disclosed embodiment, which can be called a reference entity. The reference entity can be used to further screen at least one candidate word obtained by the completion prediction. For example, the entity semantic representation of the reference entity can be used to represent the correlation between the semantic representation of the words of each candidate word, and the candidate words in the front preset order (such as the first 5) are selected in descending order of the correlation as the final candidate words for the missing position. The above method can perform entity search and completion prediction in parallel, thereby further improving the efficiency of text completion.
[0268] Step S1630: using the candidate segments at each missing position, the complete text of the text to be completed is obtained.
[0269] In an implementation scenario, a corresponding candidate word can be added to each missing position, so that several candidate texts of the text to be completed can be obtained, so that the final score of each candidate text can be obtained, and based on the final scores of the several candidate texts, a candidate text is selected as the complete text of the text to be completed. For details, please refer to the relevant description in the aforementioned disclosed embodiment, which will not be repeated here.
[0270] In another implementation scenario, in order to improve the accuracy of the final score, a corresponding candidate word can be added to each missing position to obtain several candidate texts of the text to be completed, and for each candidate text, the words in the candidate text are sorted in reverse order to obtain the reverse text of the candidate text, thereby obtaining the final score of the candidate text based on the first score of the candidate text and the second score of the reverse text, and then based on the final scores of the several candidate texts, a candidate text is selected as the complete text of the text to be completed. For details, please refer to the relevant description in the aforementioned disclosed embodiment, which will not be repeated here.
[0271] Different from the aforementioned embodiment, by obtaining the text to be completed and determining the text library from which the missing content of the text to be completed originates, the knowledge graph and text library corresponding to the preset knowledge field are used to perform completion prediction on the text to be completed, and at least one candidate word for the missing position is obtained, and then the candidate fragments of each missing position are used to obtain the complete text of the text to be completed. Therefore, the missing content of the text to be completed can be completed without relying on manual labor, which can improve the efficiency of text completion and reduce the cost of text completion. In addition, since the missing content is determined to originate from a text library involving a preset knowledge field, the knowledge graph and text library corresponding to the preset knowledge field are used to perform completion prediction, which is conducive to further improving the accuracy of text completion.
[0272] See also Fig.17 , Fig.17 1 is a schematic diagram of a framework of an embodiment of an electronic device 1700 of the present application. The electronic device 1700 includes a memory 1701 and a processor 1702 coupled to each other, the memory 1701 stores program instructions, and the processor 1702 is used to execute the program instructions to implement the steps in any of the above-mentioned text completion method embodiments. Specifically, the electronic device 1700 may include but is not limited to: a desktop computer, a laptop computer, a tablet computer, a server, etc., which are not limited here.
[0273] Specifically, the processor 1702 is used to control itself and the memory 1701 to implement the steps of any of the above-mentioned text completion method embodiments. The processor 1702 can also be called a CPU (Central Processing Unit). The processor 1702 may be an integrated circuit chip with signal processing capabilities. The processor 1702 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 1702 can be implemented by an integrated circuit chip.
[0274] In some disclosed embodiments, the processor 1702 is used to obtain the text to be completed and determine the text library from which the missing content of the text to be completed originates; wherein the text to be completed includes at least one missing position; the processor 1702 is used to use the text library to perform completion prediction on the text to be completed and obtain at least one candidate word for the missing position; the processor 1702 is used to use the candidate words for each missing position to obtain the complete text of the text to be completed.
[0275] Different from the aforementioned embodiment, by obtaining the text to be completed and determining the text library from which the missing content of the text to be completed originates, the text library is used to perform completion prediction on the text to be completed, and at least one candidate word for the missing position is obtained, and then the candidate words for each missing position are used to obtain the complete text of the text to be completed. Therefore, the missing content of the text to be completed can be completed without relying on manual labor, which can improve the efficiency of text completion and reduce the cost of text completion. In addition, since the missing content is determined to originate from the text library, the text library is used to perform completion prediction on the text to be completed, and at least one candidate word for the missing position is directly obtained, that is, the missing unknown is not limited to missing text, words or entities, which can be conducive to the prediction of mixed granularity such as text, words, and entities.
[0276] In some disclosed embodiments, the text library includes at least one reference text, and the reference text includes at least one reference word. The processor 1702 is used to perform a first semantic extraction on the text to be completed to obtain an individual semantic representation of each missing position; the processor 1702 is used to obtain at least one candidate word for the missing position using the individual semantic representation of the missing position and the word semantic representation of each reference word for each missing position.
[0277] Different from the above-mentioned embodiment, by performing semantic extraction on the text to be completed, the individual semantic representation of each missing position is obtained, so that for each missing position, the individual semantic representation of the missing position and the semantic representation of each reference word are used to directly obtain at least one candidate word for the missing position, which can help improve the accuracy and efficiency of the completion prediction. In addition, since the candidate words are directly predicted for the missing position, the missing position is not limited to missing text, words or entities, which can help to achieve the prediction of mixed granularity such as text, words, and entities.
[0278] In some disclosed embodiments, processor 1702 is used to perform word segmentation and part-of-speech tagging on at least one reference text, respectively, to obtain a number of words tagged with part-of-speech categories; processor 1702 is used to segment words whose part-of-speech categories are preset categories word by word, and use the segmented words and unsegmented words to obtain a number of reference words; processor 1702 is used to perform second semantic extraction on a number of reference words, respectively, to obtain word semantic representations of the reference words.
[0279] Different from the aforementioned embodiment, by performing word segmentation and part-of-speech tagging on at least one reference text respectively, a number of words marked with part-of-speech categories are obtained, and the words with the part-of-speech category of the preset category are segmented word by word, so as to obtain a number of reference words by using the segmented words and the unsegmented words, which can be helpful to obtain reference words with mixed granularity including text, words and entities, and can further be helpful to realize the subsequent prediction of mixed granularity such as text, words, entities, etc.
[0280] In some disclosed embodiments, processor 1702 is used to perform at least one of the following: among the segmented words and the unsegmented words, filtering out words whose occurrence frequency is higher than a preset frequency to obtain a number of reference words; among the segmented words and the unsegmented words, eliminating words whose part of speech categories meet preset elimination conditions to obtain a number of reference words.
[0281] Different from the above-mentioned embodiment, among the segmented words and the unsegmented words, words with a frequency higher than a preset frequency are selected to obtain a number of reference words, which can further reduce the scale of the reference words. And among the segmented words and the unsegmented words, words whose part of speech categories meet the preset elimination conditions are eliminated to obtain a number of reference words, which can further reduce the scale of the reference words.
[0282] In some disclosed embodiments, processor 1702 is used to fill in a default symbol at each missing position of the text to be completed to obtain the text to be processed; processor 1702 is used to perform semantic extraction on the text to be processed to obtain the individual semantic representation of each default symbol, and use the individual semantic representation of the default symbol as the individual semantic representation of the missing position where the default symbol is located.
[0283] Different from the aforementioned embodiment, a default symbol is added to each missing position of the text to be completed to obtain the text to be processed, and on this basis, semantic extraction is performed on the text to be processed to obtain the individual semantic representation of each default symbol, and the individual semantic representation of the default symbol is used as the individual semantic representation of the missing position where the default symbol is located. This can facilitate the subsequent prediction of candidate words for the missing position based on the individual semantic representation of a single default symbol. Therefore, the missing position is not limited to missing text, words or entities, which can facilitate the prediction of mixed granularities such as text, words, and entities.
[0284] In some disclosed embodiments, processor 1702 is used to obtain a predicted probability value for each reference word using individual semantic representations and word semantic representations of each reference word; processor 1702 is used to select reference words located in a front preset order as candidate words for the missing position in descending order of predicted probability values.
[0285] Different from the aforementioned embodiment, by utilizing the individual semantic representation and the word semantic representation of each reference word, the predicted probability value of each reference word is obtained, and then the reference word located in the front preset order is selected as the candidate word for the missing position in descending order of the predicted probability value, and then the reference word can be selected as the candidate word for the missing position based on the individual semantic representation and the word semantic representation, which can help improve the accuracy of the candidate words.
[0286] In some disclosed embodiments, the first semantic extraction is performed using a prediction network, which is obtained by training a preset neural network using sample text, and the sample text includes at least one sample missing position. The processor 1702 is used to segment and tag the original text to obtain a number of words marked with part-of-speech categories; the processor 1702 is used to segment the words whose part-of-speech categories are preset categories word by word, and select a preset proportion of words from the segmented words and the unsegmented words as default; the processor 1702 is used to use the original text after default as the sample text, and use the position of the defaulted word as the sample missing position.
[0287] Different from the aforementioned embodiment, by performing word segmentation and part-of-speech tagging on the original text, a number of words marked with part-of-speech categories are obtained, and the words with the part-of-speech category as the preset category are segmented word by word, so that a preset proportion of words are selected as default among the segmented words and the unsegmented words, and then the original text after the default is used as the sample text, and the position of the defaulted word is used as the sample missing position of the sample text. Therefore, it is possible to construct a sample text with missing content containing mixed granularities such as characters, words, entities, etc., which can help improve the adaptability of the prediction network obtained by subsequent training to the text to be completed with mixed granularities such as missing characters, words, entities, etc., and thus can help improve the accuracy of subsequent completion predictions.
[0288] In some disclosed embodiments, processor 1702 is used to fill in a corresponding candidate word at each missing position to obtain several candidate texts for the text to be completed; processor 1702 is used to, for each candidate text, sort the words in the candidate text in reverse order to obtain the reverse text of the candidate text, and obtain the final score of the candidate text based on the first score of the candidate text and the second score of the reverse text; processor 1702 is used to select a candidate text as the complete text of the text to be completed based on the final scores of several candidate texts.
[0289] Different from the aforementioned embodiment, a corresponding candidate word is added to each missing position to obtain several candidate texts of the text to be completed, and for each candidate text, the words in the candidate text are sorted in reverse order to obtain the reverse text of the candidate text, and thus the final score of the candidate text is obtained based on the first score of the candidate text and the second score of the reverse text. Therefore, in the process of scoring the candidate text, the forward sequence and reverse sequence of the candidate text can be comprehensively considered for scoring, which can help improve the accuracy of the final score, and further in the subsequent process of obtaining the complete text based on the final score, can help improve the accuracy of the complete text.
[0290] See also Fig.18 , Fig.18 1 is a schematic diagram of a framework of an embodiment of a storage device 1800 of the present application. The storage device 1800 stores program instructions 1801 that can be executed by a processor, and the program instructions 1801 are used to implement the steps in any of the above text completion method embodiments.
[0291] The above solution can improve the efficiency of text completion and reduce the cost of text completion.
[0292] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.
[0293] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.
[0294] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.
[0295] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
[0296] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0297] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.< / t> < / s>
Claims
1. A text completion method, characterized in that: include: Acquire a text to be completed, and determine a text library from which the missing content of the text to be completed originates; wherein the text to be completed includes at least one missing position, the text library includes at least one reference text, and the reference text includes at least one reference word; Performing a first semantic extraction on the text to be completed to obtain individual semantic representations of each missing position, and, Performing word segmentation and part-of-speech tagging on the at least one reference text respectively to obtain a number of words tagged with part-of-speech categories; segmenting the words whose part-of-speech categories are preset categories word by word, and using the segmented words and the unsegmented words to obtain a number of reference words of multiple mixed granularities; performing second semantic extraction on the reference words respectively to obtain word semantic representations of the reference words; The first semantic extraction is performed using a prediction network, which is obtained by training a preset neural network using sample text, and the sample text includes at least one sample missing position; the sample text acquisition step includes: segmenting and part-of-speech tagging the original text to obtain a number of words tagged with part-of-speech categories; segmenting the words whose part-of-speech categories are preset categories word by word, and selecting a preset proportion of words from the segmented words and the unsegmented words to be omitted; using the original text after the omission as the sample text, and using the position of the omitted word as the sample missing position; For each missing position, using the individual semantic representation of the missing position and the word semantic representation of each reference word, obtaining at least one candidate word for the missing position; Using the candidate words at each of the missing positions to obtain the complete text of the text to be completed includes: Fill in a corresponding candidate word in each missing position to obtain a plurality of candidate texts of the text to be completed; For each candidate text, the words in the candidate text are sorted in reverse order to obtain a reverse text of the candidate text, and a final score of the candidate text is obtained based on the first score of the candidate text and the second score of the reverse text; Based on the final scores of the plurality of candidate texts, one candidate text is selected as the complete text of the text to be completed.
2. The method according to claim 1, characterized in that The method uses the segmented words and the unsegmented words to obtain a plurality of reference words, including at least one of the following: Selecting words with a frequency higher than a preset frequency from the segmented words and the unsegmented words to obtain the reference words; Among the segmented words and the unsegmented words, the words whose part of speech categories meet the preset elimination conditions are eliminated to obtain the reference words.
3. The method according to claim 1, characterized in that The first semantic extraction is performed on the text to be completed to obtain individual semantic representations of each missing position, including: At each missing position of the text to be completed, a default symbol is respectively added to obtain the text to be processed; Semantic extraction is performed on the text to be processed to obtain individual semantic representations of each of the default symbols, and the individual semantic representations of the default symbols are used as the individual semantic representations of the missing positions where the default symbols are located.
4. The method according to claim 1, characterized in that: The step of obtaining at least one candidate word for the missing position by using the individual semantic representation of the missing position and the word semantic representation of each of the reference words comprises: Using the individual semantic representation and the word semantic representation of each of the reference words, obtaining a predicted probability value of each of the reference words; According to the order of the predicted probability values from large to small, the reference words at the front preset positions are selected as the candidate words for the missing positions.
5. An electronic device, characterized in that: It comprises a memory and a processor coupled to each other, wherein the memory stores program instructions, and the processor is used to execute the program instructions to implement the text completion method according to any one of claims 1 to 4.
6. A storage device, characterized in that: Program instructions that can be executed by a processor are stored, and the program instructions are used to implement the text completion method described in any one of claims 1 to 4.
