Natural language processing method and model acquisition method and device thereof, storage medium

Abstract

The application discloses a natural language processing method and an acquisition method, device and storage medium of a model thereof, and relates to the technical field of artificial intelligence. In the method, a target text obtained is input into a target natural language processing model to obtain a processing result of the natural language. Since the target natural language processing model is obtained by updating a basic processing model based on entity prediction loss values and entity relationship prediction loss values of the basic processing model, the target natural language processing model can not only better understand the entity itself, but also better understand the complex interaction relationship between different entities, and the result accuracy of the natural language processing by using the target natural language processing model is higher, and the reliability is better.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a natural language processing method and a method, apparatus, and storage medium for acquiring and storing its model. Background Technology

[0002] Natural language processing (NLP) is an important research direction in the field of artificial intelligence (AI). Natural language refers to scientific language that includes entities, and NLP can include identifying entities and extracting the relationship between two entities.

[0003] Currently, natural language processing is generally performed using models such as bidirectional encoder representation from transformers (BERT). However, current processing models only focus on the entities themselves during natural language processing, ignoring the complex interactions between entities in a document. This results in low processing accuracy and poor reliability. Summary of the Invention

[0004] This application provides a natural language processing method, a method for obtaining the model, and a storage medium, which can solve the problems of low accuracy and poor reliability in natural language processing in related technologies.

[0005] The technical solution is as follows: On the one hand, a natural language processing method is provided, the method comprising: Obtain the target text, which includes at least one entity; A target natural language processing model is obtained, wherein the target natural language processing model is a model obtained by updating the basic processing model based on the entity prediction loss value and the entity relationship prediction loss value of the basic processing model. The entity prediction loss value is determined based on the similarity of each entity predicted by the basic processing model, and the entity relationship prediction loss value is determined based on the relationship similarity of each entity pair predicted by the basic processing model. Each entity pair includes two entities. The target text is input into the target natural language processing model to obtain the processing result output by the target natural language processing model.

[0006] On the other hand, a method for obtaining a natural language processing model is provided, the method comprising: Acquire multiple first training samples and multiple second training samples, each first training sample including one entity pair, each second training sample including two entity pairs, the entity pair including two entities; The plurality of first training samples are input into the basic processing model to obtain the entity prediction loss value of the basic processing model. The entity prediction loss value is determined based on the similarity between two entities in each of the first training samples predicted by the basic processing model. The multiple second training samples are input into the basic processing model to obtain the entity relationship prediction loss value of the basic processing model. The entity relationship prediction loss value is determined based on the relationship similarity between two entity pairs in each of the second training samples predicted by the basic processing model. Based on the entity prediction loss value and the entity relationship prediction loss value, the parameters of the basic processing model are updated to obtain the target natural language processing model.

[0007] In another aspect, a natural language processing apparatus is provided, the apparatus comprising: A text acquisition module is used to acquire target text, wherein the target text includes at least one entity; The model acquisition module is used to acquire a target natural language processing model. The target natural language processing model is a model obtained by updating the basic processing model based on the entity prediction loss value and the entity relationship prediction loss value of the basic processing model. The entity prediction loss value is determined based on the similarity of each entity predicted by the basic processing model, and the entity relationship prediction loss value is determined based on the relationship similarity of each entity pair predicted by the basic processing model. Each entity pair includes two entities. A processing model is used to input the target text into the target natural language processing model and obtain the processing result output by the target natural language processing model.

[0008] Furthermore, a device for acquiring a natural language processing model is provided, the device comprising: The sample acquisition module is used to acquire multiple first training samples and multiple second training samples. Each first training sample includes one entity pair, and each second training sample includes two entity pairs, wherein each entity pair includes two entities. A first input model is used to input the plurality of first training samples into a basic processing model to obtain an entity prediction loss value of the basic processing model. The entity prediction loss value is determined based on the similarity between two entities in each of the first training samples predicted by the basic processing model. The second input model is used to input the plurality of second training samples into the basic processing model to obtain the entity relationship prediction loss value of the basic processing model. The entity relationship prediction loss value is determined based on the relationship similarity between two entity pairs in each of the second training samples predicted by the basic processing model. The update module is used to update the parameters of the basic processing model based on the entity prediction loss value and the entity relationship prediction loss value, so as to obtain the target natural language processing model.

[0009] Optionally, the plurality of first training samples include: M first positive training samples obtained from a plurality of first documents included in the first document set, and a plurality of first negative training samples corresponding to each first positive training sample; wherein, each first positive training sample includes: a head entity in a first document, and a tail entity of the head entity, and each first negative training sample includes: a head entity in the first positive training sample corresponding to the first negative training sample, and a reference entity in the first document to which the head entity belongs, wherein the reference entity is an entity in the first document other than the tail entity, and M is an integer greater than 1; The plurality of second training samples include: N second positive training samples obtained from a plurality of second documents included in the second document set, and a plurality of second negative training samples corresponding to each second positive training sample; wherein, each second positive training sample includes: two entity pairs with the same relationship obtained from at least one second document, each second negative training sample includes: two entity pairs with different relationships obtained from at least one second document, and two entity pairs with no relationship, and one entity pair in each second negative training sample is the same as one entity pair in a second positive training sample corresponding to the second negative training sample, and N is an integer greater than 1.

[0010] Optionally, the first input module is used to input each first positive training sample and the corresponding plurality of first negative training samples into the basic processing model to obtain the entity prediction loss value of the basic processing model. The second input module is used to input each second positive training sample and the corresponding plurality of second negative training samples into the basic processing model to obtain the entity relationship prediction loss value of the basic processing model.

[0011] Optionally, the entity prediction loss value Lep of the basic processing model obtained based on the m-th first positive training sample and the corresponding multiple first negative training samples satisfies: ; in, It refers to the similarity between the head entity and the tail entity in the m-th first positive training sample. It refers to the similarity between the head entity and the reference entity included in the j-th first negative training sample among the multiple first negative training samples corresponding to the m-th first positive training sample, where I refers to the number of the multiple first negative training samples, and m is an integer less than or equal to M, and j is an integer less than or equal to I. The entity relationship prediction loss value Lrp of the basic processing model obtained based on the nth second positive training sample and the corresponding multiple second negative training samples satisfies: ; in, It refers to the relational similarity between two entity pairs in the nth second positive training sample. Nneg refers to the relational similarity between two entity pairs included in the kth second negative training sample among the multiple second negative training samples corresponding to the nth second positive training sample. Nneg refers to the number of the multiple second negative training samples, where n is an integer less than or equal to N and k is an integer less than or equal to Nneg.

[0012] Optionally, the update module is used for: The basic processing model is updated based on the sum of the target loss value, the entity prediction loss value, and the entity relationship prediction loss value until the update termination condition is met. The base processing model that meets the update termination condition is determined as the target natural language processing model; The target loss value is used to indicate the computational loss of the basic processing model itself.

[0013] In another aspect, a computer device is provided, the computer device including a processor and a memory, the memory storing at least one piece of program code, the at least one piece of program code being loaded and executed by the processor to implement the natural language processing method as described above, or to implement the natural language processing model acquisition method as described above.

[0014] In another aspect, a computer-readable storage medium is provided, wherein at least one piece of program code is stored therein, the at least one piece of program code being loaded and executed by a processor to implement the natural language processing method as described above, or to implement the method for obtaining a natural language processing model as described above.

[0015] The beneficial effects of the technical solutions provided in this application embodiment can include at least: This application provides a natural language processing method and a method, apparatus, and storage medium for acquiring the same model. The method inputs the acquired target text into a target natural language processing model to obtain the natural language processing result. Since the target natural language processing model is obtained by optimizing the basic processing model based on the entity prediction loss value and entity relationship prediction loss value, it not only has a better understanding of the entities themselves but also a better understanding of the complex interaction relationships between different entities. Therefore, the results of natural language processing using this target natural language processing model have higher accuracy and better reliability. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 These are schematic diagrams illustrating the implementation environments involved in the various embodiments of this application; Figure 2 This is a flowchart of a natural language processing method provided in an embodiment of this application; Figure 3 This is a flowchart of a method for obtaining a natural language processing model provided in an embodiment of this application; Figure 4 This is a flowchart of another method for obtaining a natural language processing model provided in an embodiment of this application; Figure 5 This is a schematic diagram of entity prediction training provided in an embodiment of this application; Figure 6 This is a schematic diagram of entity relationship prediction training provided in an embodiment of this application; Figure 7 This is a structural block diagram of a natural language processing device provided in an embodiment of this application; Figure 8 This is a structural block diagram of a natural language processing model acquisition device provided in an embodiment of this application; Figure 9 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0019] First, some proprietary terms involved in the embodiments of this application are explained as follows: Artificial intelligence (AI) is the theory, methods, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to have perception, reasoning, and decision-making capabilities. AI technology is a comprehensive discipline involving a wide range of fields, encompassing both hardware and software technologies. Basic AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operating / interactive systems, and mechatronics. AI software technologies mainly include computer vision, speech processing, natural language processing, and machine learning / deep learning. The solutions provided in the following embodiments of this application relate to natural language processing and machine learning technologies in artificial intelligence.

[0020] Machine learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory, among others. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instructional learning.

[0021] Natural Language Processing (NLP) is an important field within computer science and artificial intelligence. It studies the theories and methods for enabling effective communication between humans and computers using natural language. NLP is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field involves natural language—the language people use in daily life—and thus it has a close relationship with linguistic research. NLP technologies typically include text processing, semantic understanding, machine translation, question answering, and knowledge graphs.

[0022] Figure 1 This is a schematic diagram of the implementation environment involved in the following embodiments of this application. For example... Figure 1As shown, the implementation environment may include a terminal 01 and a server 02, which can establish a communication connection through a wired or wireless network. The methods described in the following embodiments can be applied to the terminal 01, the server 02, or both.

[0023] For example, the natural language processing method described in the following embodiments can be applied to terminal 01, and the natural language processing model acquisition method can be applied to server 02. That is, server 02 can first use the natural language processing model acquisition method described in the following embodiments to acquire the target natural language processing model, and send the acquired target natural language processing model to terminal 01, and then terminal 01 can use the target natural language processing model to perform natural language processing.

[0024] Optionally, terminal 01 can be any electronic product capable of human-computer interaction with the user through a keyboard, touchpad, touchscreen, remote control, voice interaction, or handwriting device. For example, Figure 1 The desktop computer, personal computer (PC), mobile phone, wearable device, or tablet computer shown are examples. Server 02 can be... Figure 1 The server shown can be a single server, a server cluster consisting of multiple servers, or a cloud computing service center. Furthermore, those skilled in the art should understand that the terminal 01 and server 02 described above are merely examples, and other existing or future terminals or servers that are applicable to this application should also be included within the scope of protection of this application.

[0025] Figure 2 This is a flowchart of a natural language processing method provided in an embodiment of this application. This method can be applied to... Figure 1 In terminal 01 shown. For example... Figure 2 As shown, the method may include: Step 201: Obtain the target text.

[0026] Optionally, the terminal may receive target text input by the user, or it may actively invoke the target text for processing in response to a received processing instruction. Furthermore, the target text obtained by the terminal may be a sentence, a paragraph containing multiple sentences, or a document containing many sentences. In addition, in this embodiment, the target text obtained by the terminal may include at least one entity.

[0027] Entities, also known as named entities, refer to words or phrases in text that have specific meanings. Entities can generally be divided into three main categories (entity, time, and number) and seven subcategories (person names, place names, organization names, time, date, currency, and percentage).

[0028] Furthermore, two entities can form an entity pair. Two different entities can have a relationship in some way, and one of the entities with the relationship can be called the head entity, and the other entity can be called the tail entity. Triples can be constructed to represent two entities and their relationship; that is, a triple can include: (head entity, relationship, tail entity). For example, for the target text "Beijing is the capital of China," the text includes a triple (China, capital, Beijing). Combining this with the above description of entities, we can determine that the target text includes two entities: the head entity "China" and the tail entity "Beijing," with "capital" representing the relationship between the two entities.

[0029] The above embodiments also demonstrate that accurately identifying entities within a text is crucial for capturing the facts of that text, especially in applications where the text consists of documents containing multiple sentences. If the facts of a text can be reliably captured, the results of natural language processing will also be more accurate.

[0030] Step 202: Obtain the target natural language processing model.

[0031] In this application, the target natural language processing model can be a model obtained by updating the basic processing model based on the entity prediction loss value and entity relationship prediction loss value of the basic processing model.

[0032] The entity prediction loss value can be determined based on the similarity of each entity predicted by the basic processing model, and the entity relationship prediction loss value can be determined based on the relationship similarity of each entity pair predicted by the basic processing model. Each entity pair can include two entities, such as a head entity and a tail entity.

[0033] Optionally, the underlying processing model can be any model with natural language processing capabilities. For example, the BERT model or a BERT model based on coreferential resolution, namely the CorefBERT model.

[0034] However, the existing basic processing models all neglect the complex interactions between entities when performing natural language processing, resulting in poor accuracy of the processing results. In this application, the embodiment updates (or optimizes) the basic processing model based on the entity prediction loss value and entity relationship prediction loss value, enabling the updated basic processing model—that is, the target natural language processing model described in this application—to better understand the entities themselves and the relationships between different entities.

[0035] Step 203: Input the target text into the target natural language processing model and obtain the processing result output by the target natural language processing model.

[0036] In this embodiment, after obtaining the target text and the target natural language processing model, the terminal can directly input the target text into the target natural language processing model to obtain the processing result and complete the natural language processing task. Furthermore, based on the description of step 202 above, since the target natural language processing model obtained by the terminal in this embodiment has a relatively accurate understanding of entities and entity relationships, the final processing result obtained in this embodiment has good accuracy and high reliability.

[0037] Optionally, based on the above explanation of the terminology of natural language processing, the processing results of natural language processing can include at least one of the following results: the category of at least one entity in the target text, the relationship of at least one entity pair in the target text, the question-and-answer results of the target text, and the translation results of the target text.

[0038] Here, "category" can refer to the classification to which an entity belongs. For example, for the entity "Beijing" mentioned above, its category could be "city name." "Entity pair relationship" can refer to the relationship between the two entities in the entity pair. For example, for the entities "Beijing" and "China" mentioned above, their relationship could be "capital." "Question-answering result" can refer to a response to the target text; correspondingly, the target text needs to include at least one question. "Translation result" can refer to an interpretation and translation of the target text, such as translating the target text from Chinese to English. Of course, the processing result is not limited to the above examples; it can also include the speech recognition result of the target text. Accordingly, the target text obtained by the terminal in step 201 above can be in speech form.

[0039] In summary, this application provides a natural language processing method that inputs acquired target text into a target natural language processing model to obtain natural language processing results. Since this target natural language processing model is obtained by optimizing the basic processing model based on the entity prediction loss value and entity relationship prediction loss value, it not only has a better understanding of the entities themselves but also a better understanding of the complex interaction relationships between different entities. Therefore, the results of natural language processing using this target natural language processing model have higher accuracy and better reliability.

[0040] As can be seen from the above embodiments, in order to obtain the target natural language processing model, it is first necessary to call a basic processing model, and the basic processing model can be a natural language processing model.

[0041] For example, the base processing model can be the BERT model, or an improved BERT model (RoBERTa model), or the CorefBERT model, or a convolutional neural network (CNN) model, or a deep neural network (DNN) model. If it's a BERT model, the base model, BERT_BASE, can be used. This BERT_BASE model typically includes 12 transformer layers and has a total storage of approximately 110 MB. Of course, the above models are merely illustrative; any language processing model capable of machine learning can be used as the base processing model.

[0042] The following describes the method for obtaining the target natural language processing model described in the above embodiments: Figure 3 This is a flowchart illustrating a method for obtaining a natural language processing model according to an embodiment of this application. This method can be applied to... Figure 1 In server 02 shown. For example... Figure 3 As shown, the method may include: Step 301: Obtain multiple first training samples and multiple second training samples.

[0043] Optionally, in this embodiment of the application, the server may first obtain a large number of documents, and then extract multiple first training samples and multiple second training samples from the large number of documents to complete the following training.

[0044] The first training sample can be a sample used to train the basic processing model for entity prediction; correspondingly, each first training sample can include one entity pair. The second training sample can be a sample used to train the basic processing model for entity relationship prediction; correspondingly, each second training sample can include two entity pairs. Furthermore, as described in the above embodiments, an entity pair can include two entities.

[0045] Step 302: Input multiple first training samples into the basic processing model to obtain the entity prediction loss value of the basic processing model.

[0046] After acquiring multiple initial training samples, the server can sequentially input these samples into the base processing model in batches to determine the entity prediction loss value for each initial training sample processed by the base processing model. Furthermore, this entity prediction loss value can be determined by the server based on the similarity between two entities in each initial training sample predicted by the base processing model.

[0047] Step 303: Input multiple second training samples into the basic processing model to obtain the entity relationship prediction loss value of the basic processing model.

[0048] After acquiring multiple second training samples, the server can sequentially input these samples into the base processing model in batches to determine the entity relationship prediction loss value for each second training sample processed by the base processing model. Furthermore, this entity relationship prediction loss value can be determined by the server based on the relationship similarity between two entity pairs in each second training sample predicted by the base processing model.

[0049] Step 304: Based on the entity prediction loss value and the entity relationship prediction loss value, update the parameters of the basic processing model to obtain the target natural language processing model.

[0050] Optionally, the server can update and optimize the parameters of the basic processing model after each input training sample to obtain the entity prediction loss value and entity relationship prediction loss value, thereby reducing the prediction loss of the basic processing model and improving its processing performance. Furthermore, the server can stop updating the basic processing model in response to a pre-set update termination condition and use the parameters of the finally updated basic processing model as the target natural language processing model. This target natural language processing model can then be used to perform the aforementioned operations. Figure 2 The natural language processing method described in the illustrated embodiment.

[0051] In summary, this application provides a method for obtaining a natural language processing model. This method can determine the entity prediction loss value and entity relationship prediction loss value of the basic processing model based on the obtained training samples, and can update and optimize the basic processing model based on these two loss values ​​to obtain the target natural language processing model. Thus, the final target natural language processing model not only has a good understanding of the entities themselves, but also a good understanding of the complex interaction relationships between different entities; that is, the target natural language processing model obtained by this application has good performance.

[0052] Figure 4 This is a flowchart illustrating another method for obtaining a natural language processing model provided in this application embodiment. This method can be applied to... Figure 1 In server 02 shown. For example... Figure 4 As shown, the method may include: Step 401: Obtain multiple first training samples.

[0053] Obtaining training samples is the foundation for model updates and optimizations. In this embodiment, the server can gradually obtain multiple first training samples for training the basic processing model to predict entities in the following manner, and each obtained first training sample may include an entity pair.

[0054] First, the server can use a web crawler to retrieve multiple documents from a knowledge base containing a large corpus (i.e., language materials). To ensure the reliability of subsequent training, the retrieved documents can include a large number of entities and a large number of entity pairs with different relationships. Furthermore, during the document retrieval process, the server can directly annotate the entities included in each document and the relationships between entity pairs within the document, thus directly identifying each triplet included in each document. Second, the server can extract a subset of documents (also called the first document set) from the retrieved documents to form the first document set, and then extract multiple first training samples from each of the first documents included in this first document set.

[0055] For example, consider a knowledge base called AA. First, the server can use a web crawler to retrieve a large number of documents from AA, and can use an entity recognition tool (such as spaCy, where spaCy is the tool's identifier) ​​to identify the entities in each document. Then, the server can link the identified entities and the hyperlinks provided by AA to the AA data, obtaining the corresponding knowledge base (KB) identification ID, i.e., the KB ID. Finally, the server can further determine the relationships between different entities by querying the AA data. In this way, entity pairs and relationships in multiple documents can be identified. Furthermore, the server can also use remote supervision techniques to determine the relationship between two entities.

[0056] For example, let's take a scenario where 1 million documents were acquired, each document annotated with over 4,000 relationships, and each document contains an average of 186.9 words, 7.2 relationships, and 12.9 entities, with each entity appearing an average of 1.3 times. Based on this number of documents, the server can extract a portion (e.g., 500,000) of these 1 million documents as a first document set, and further extract multiple first training samples from this set.

[0057] Optionally, in this embodiment of the application, the multiple first training samples obtained by the server may include: M first positive training samples obtained from multiple first documents included in the first document set, and multiple first negative training samples corresponding to each first positive training sample, where M can be a positive integer greater than 1.

[0058] Each first positive training sample may include: a head entity in a first document, and a tail entity of the head entity. Each first negative training sample may include: the head entity in the first positive training sample corresponding to the first negative training sample, and a reference entity in the first document to which the head entity belongs, which may be an entity in the first document other than the tail entity.

[0059] Example, reference Figure 5The document shows a schematic diagram of a first document, the contents of which are: "Culiacán is a city in northwestern Mexico. Culiacán is the capital of the state of Sinaloa. Culiacán is also the seat of Culiacán Municipality. Culiacán is in the center of Sinaloa, at about the same distance to the two other urban centers of Sinaloa: Los Mochis to the north and Mazatlán to the north." The Chinese translation is: "(Translation: Culiacán is a city in northwestern Mexico. Culiacán is the capital of the state of Sinaloa. Culiacán is also the seat of Culiacán Municipality. Culiacán is in the center of Sinaloa, at about the same distance to the two other urban centers of Sinaloa: Los Mochis to the north and Mazatlán to the north.)" Furthermore, the first document is labeled with the tail entity "Mexico" and the head entity "Sinaloa," while "Culiacán," "Los Mochis," "Mazatlán," and "Municipality" are all reference entities. Therefore, a first positive training sample extracted by the server from this first document could be "Mexico" and "Sinaloa." Corresponding to this first positive training sample are four first negative training samples: "Sinaloa" and "Culiacán," "Sinaloa" and "Los Mochis," "Sinaloa" and "Mazatlán," and "Sinaloa" and "Municipality." Figure 5 In the text, bold, underlined, and italic markers represent tail entities; bold and underlined markers represent relationships; italic and underlined markers represent head entities; and only bold markers represent reference entities.

[0060] It should be noted that the server may input the first document into the basic processing model and then extract the first training sample, or the server may first extract multiple first training samples and then input the first training samples into the basic processing model, that is, perform the operation of step 403 below.

[0061] Step 402: Obtain multiple second training samples.

[0062] The method for obtaining the second training samples can be found in step 401. The difference is that the server can use the documents other than the first document (also called the second documents) from the multiple documents obtained as the second document set, and extract multiple second training samples from the multiple second documents included in this second document set. For example, assuming that a total of 1 million documents are obtained, and the server extracts 50 documents as the first document set, then the server can directly use the remaining 500,000 documents as the second document set.

[0063] Optionally, the multiple second training samples obtained by the server may include: N second positive training samples obtained from multiple second documents included in the second document set, and multiple second negative training samples corresponding to each second positive training sample, where N can be an integer greater than 1.

[0064] Each second positive training sample may include two entity pairs with the same relationship obtained from at least one second document (i.e., one or more second documents). Each second negative training sample may include two entity pairs with different relationships obtained from at least one second document, and two entity pairs with no relationship (i.e., no-relationship labels), and one entity pair in each second negative training sample may be the same as one entity pair in a corresponding second positive training sample. By adding unrelated entity pairs as second negative training samples, the diversity of learning can be enhanced, the utilization rate of negative samples can be effectively improved, and the update effect on the basic processing model can be optimized.

[0065] Example, reference Figure 6It shows a schematic diagram of the three documents D1, D2 and D3 that were obtained. Document D1 includes “...Since 1773, when the Royal Swedish Opera was founded by Gustav III of Sweden…”, translated into Chinese as “...Since 1773 when Gustav III of Sweden founded the Royal Swedish Opera…”; Document D2 includes “...Gates is an American business magnate, software developer, and philanthropist… He left his board positions at Microsoft…”, translated into Chinese as “...Gates is an American business magnate, software developer, and philanthropist… He left his board positions at Microsoft…”; Document D3 includes “...Samarinda is the capital of East Kalimantan, Indonesia, on the island of Borneo…Samarinda is known for its traditional food amplang, as well as the cloth Sarung Samarinda…”, translated into Chinese as “...Samarinda is the capital of East Kalimantan province in Indonesia, located on the island of Borneo…Samarinda is known for its traditional food amplang and cloth Sarung Samarinda…”. Furthermore, in document D1, the entity pair includes the header entity "oyal Swedish Opera" and the tail entity "Gustav III of Sweden," with a relationship of "founded by." In document D2, the entity pair includes the header entity "Microsoft" and the tail entity "Gates," with a relationship of "founded by." In document D3, one entity pair includes the header entity "East Kalimantan" and the tail entity "Samarinda," with a relationship of "capital of," while another entity pair includes the header entity "Indonesia" and the tail entity "SarungSamarinda," with a relationship of "country of." Figure 6 In the text, bold, underlined, and italic text indicates a tail entity; bold and / or underlined text indicates a relationship; and italic and underlined text indicates a head entity.

[0066] Therefore, the second positive training sample extracted by the server from these three second documents can be: entity pairs (oyal Swedish Opera, Gustav III of Sweden) and entity pairs (Microsoft, Gates) with the same relation "founded by". Furthermore, the server can obtain two second negative training samples corresponding to these second positive training samples. One second negative training sample consists of entity pairs (oyal Swedish Opera, Gustav III of Sweden) and entity pairs (East Kalimantan, Samarinda) with different relations, and the other second negative training sample consists of entity pairs (oyal Swedish Opera, Gustav III of Sweden) and entity pairs (Indonesia, Sarung Samarinda) with different relations.

[0067] It should be noted that the server may input the second document into the basic processing model and then extract the second training sample, or the server may first extract multiple second training samples and then input the second training samples into the basic processing model, that is, perform the operation of step 403 below.

[0068] It should also be noted that after obtaining multiple documents, the server can simultaneously obtain multiple first training samples and multiple second training samples, that is, execute steps 401 and 402 above at the same time.

[0069] Step 403: Input each first positive training sample and the corresponding multiple first negative training samples into the basic processing model to obtain the entity prediction loss value of the basic processing model.

[0070] In this embodiment, the server can sequentially input each of the multiple first training samples obtained, i.e., each first positive training sample and its corresponding multiple first negative training samples, into the basic processing model. Furthermore, each input yields an entity prediction loss value for the basic processing model.

[0071] The entity prediction loss value is used to indicate the loss of the basic processing model in predicting entities, and can be used to measure the performance of the basic processing model in predicting entities. Generally speaking, the smaller the entity prediction loss value, the better the entity prediction performance of the basic processing model, that is, the higher the prediction accuracy.

[0072] Optionally, in this embodiment, the entity prediction loss value Lep of the basic processing model obtained by the server based on the m-th first positive training sample and the corresponding multiple first negative training samples among the M first training samples can satisfy: Formula (1); in, It could refer to the similarity between the head entity and the tail entity in the m-th first positive training sample. It can refer to the similarity between the head entity and the reference entity included in the j-th first negative training sample among multiple first negative training samples corresponding to the m-th first positive training sample. I can refer to the number of multiple first negative training samples, and m can be an integer less than or equal to M, and j can be an integer less than or equal to I. Since formula (1) is used to calculate the loss value, formula (1) can also be called the loss function for entity prediction.

[0073] Optionally, similarity can be represented using cosine similarity, or it can be represented using other values ​​that indicate similarity, such as Euclidean distance. If cosine similarity is used, then the above... This can satisfy: Formula (2); Where em1 can refer to the vector representation of the head entity in the m-th first positive training sample, em2 can refer to the vector representation of the tail entity in the m-th first positive training sample, and τ can refer to the temperature hyperparameter, such as τ can generally be 1.05. The similarity between the head entity and the reference entity included in the j-th first negative training sample can be calculated with reference to the above formula (2), and will not be repeated here. Optionally, the vector representation of an entity can be: the average value of the vectors of all characters included in the entity.

[0074] For example, refer to Figure 5 It can be seen that the server inputs the header entity "Sinaloa" and the tail entity "Mexico" from the first document as the first positive training samples into the basic processing model 00. Furthermore, the header entity and four other reference entities (excluding the tail entity) are input into the basic processing model 00 as the corresponding four first negative training samples: "Sinaloa" and "Culiacán", "Sinaloa" and "Los Mochis", "Sinaloa" and "Mazatlán", and "Sinaloa" and "Municipality". Thus, after the basic processing model 00 calculates and processes the data based on the above formulas (1) and (2), the entity prediction loss value can be obtained.

[0075] In addition, combined Figure 5As described in the above embodiments, the principle for determining the entity prediction loss value is as follows: The basic processing model can concatenate the head entity and its relation with the known head and tail entities in a first document before the document itself. This allows the basic processing model to learn the head entity and its corresponding tail entity to be closer, i.e., more similar, through comparative learning. Furthermore, it learns the head entity to be further away from other entities in the first document besides the tail entity (i.e., reference entities). In other words, this allows the basic processing model to better understand the entity. The reference entity... Figure 5 The code involved in performing the splicing operation is "[CLS]+relationship+header entity+[SEP]+first document+[SEP]".

[0076] Step 404: Input each second positive training sample and the corresponding multiple second negative training samples into the basic processing model to obtain the entity relationship prediction loss value of the basic processing model.

[0077] In this embodiment, the server can sequentially input multiple second training samples (i.e., each second positive training sample and its corresponding multiple second negative training samples) into the basic processing model. Furthermore, each input yields an entity relationship prediction loss value for the basic processing model.

[0078] The entity relationship prediction loss value is used to indicate the loss of the basic processing model in predicting entity relationships, and can be used to measure the performance of the basic processing model in predicting entity relationships. Generally speaking, the smaller the loss value, the better the entity relationship prediction performance of the basic processing model, that is, the higher the prediction accuracy.

[0079] Optionally, in this embodiment, the entity relationship prediction loss value Lrp of the basic processing model obtained by the server based on the nth second positive training sample and the corresponding multiple second negative training samples among the N second training samples can satisfy: Formula (3); in, It can refer to the relational similarity between two entity pairs in the nth second positive training sample. Nneg refers to the relational similarity between two entity pairs included in the k-th second negative training sample among multiple second negative training samples corresponding to the n-th second positive training sample. Nneg can refer to the number of multiple second negative training samples, and n can be an integer less than or equal to N, and k can be an integer less than or equal to Nneg. Nneg can also be called a hyperparameter. Since formula (3) is used to calculate the loss value, formula (3) can also be called the loss function for entity relation prediction.

[0080] Optionally, if the similarity of relationships is still represented by cosine values, then the above... It can satisfy: Formula (4); Here, rn1 can refer to the relation vector representation of an entity in the nth second positive training sample to n1, and rn2 can refer to the relation vector representation of another entity in the nth second positive training sample to n2. The calculation can be referred to the above formula (4), and will not be repeated here.

[0081] Optionally, the relation vector representation of each entity pair can be: the sum of the representation vectors of the two entities included in the entity pair.

[0082] For example, refer to Figure 6 It can be seen that the server inputs the entity pairs (oyalSwedish Opera, Gustav III of Sweden) and (Microsoft, Gates) with the same relationship from the three documents as the second positive training samples into the basic processing model 00. Furthermore, two entity pairs with different relationships are input into the basic processing model 00 as the corresponding two first negative training samples: (oyal Swedish Opera, Gustav III of Sweden) and (East Kalimantan, Samarinda), and (oyalSwedish Opera, Gustav III of Sweden) and (Indonesia, Sarung Samarinda). Thus, after the basic processing model 00 calculates and processes the results based on the above formulas (3) and (4), an entity relationship prediction loss value can be obtained.

[0083] Similar to step 403 above, the principle for determining the entity relationship prediction loss value is to enable the basic processing model to learn to be closer (i.e., more similar) two entity pairs with the same relationship known in at least one second document, and to learn to be further apart (i.e., more distant) two entity pairs with different relationships in at least one second document. In other words, to enable the basic processing model to better understand the relationship between entity pairs.

[0084] It should be noted that the server can execute steps 403 and 404 simultaneously, that is, simultaneously determine the entity prediction loss value of the basic processing model and the entity relationship prediction loss value of the basic processing model.

[0085] Step 405: Based on the sum of the target loss value, the entity prediction loss value, and the entity relationship prediction loss value, update the basic processing model until the update termination condition is met.

[0086] Optionally, in this embodiment, in addition to obtaining the entity prediction loss value and entity relationship prediction loss value of the basic processing model, the server can also determine its own loss when processing language based on the target loss model. For example, the target loss model can be a masked language model (MLM). This can prevent the server from forgetting information and ensure processing reliability.

[0087] Furthermore, in conjunction with the descriptions of steps 403 and 404 above, since the server has acquired a large number of first training samples and a large number of second training samples, the server can update and adjust the parameters of the basic processing model once after obtaining the entity prediction loss value of the basic processing model based on each first training sample and the entity relationship prediction loss value of the basic processing model based on each second training sample, so as to optimize the prediction ability of the basic processing model, that is, reduce the entity prediction loss value and entity relationship prediction loss value of the optimized basic processing model and improve the prediction ability of the basic processing model.

[0088] Optionally, after obtaining each entity prediction loss value and entity relationship prediction loss value, the server can add the target loss value and the two predicted loss values, and update and adjust the parameters of the basic processing model based on the sum (i.e., the total loss value) until the update termination condition is met, and then stop updating, that is, stop inputting training samples (including the first training sample and the second training sample) into the basic processing model.

[0089] For example, assuming the target loss value is represented by Lmlm, the total loss value L of the basic processing model can satisfy: L=Lmlm+Lep+Lrp, and L can also be called the total loss function.

[0090] Optionally, the update termination condition can be pre-set in the server, and the update termination condition includes, but is not limited to, the following three situations: (1) the number of updates (i.e., iterative training) reaches the number threshold. (2) the total loss value is less than the loss threshold. (3) the total loss value L converges. Among them, the number threshold and the loss threshold can be set according to experience, or can be flexibly adjusted according to the application scenario. This application embodiment does not limit this. Convergence can be understood as: as the number of iterative training increases, in the training results of the target number of times, the fluctuation range of the total loss function is within the target range.

[0091] As can be seen from the above embodiments, when updating the basic processing model, not only the processing loss value of the basic processing model itself is considered, but also the loss value of the basic processing model in predicting entities and the loss value in predicting entity relationships are also considered. Therefore, the updated basic processing model can have good performance in terms of its own processing, entity prediction and entity relationship prediction.

[0092] Step 406: Determine the basic processing model that meets the update termination condition as the target natural language processing model.

[0093] Optionally, when the server detects that the update termination condition is met, it can end the operation of updating the basic processing model and determine the currently updated basic processing model as the target natural language processing model to execute the above. Figure 2 The natural language processing method described in the embodiments.

[0094] To demonstrate the processing accuracy of the target natural language processing model obtained by the server, different numbers of training samples were sampled, and the performance was validated on document-level relation extraction, document-level reading comprehension, and entity category determination tasks. The validation results are shown in Tables 1 to 3 below.

[0095] In Table 1, 10%, 50%, and 100% refer to the number of input samples. For example, 10% means that 10% of the samples were randomly selected from all the obtained samples. Assuming a total of 100 samples were obtained, then 10% of the samples would be 10 samples randomly selected from those 100 samples. Dev refers to the training samples, Test refers to the test samples, F1 is a representation used to measure the processing results, and LgF1 is another representation used to measure the processing results. Furthermore, Table 1 shows the processing results of traditional models: CNN model and Bi-directional Long Short-Term Memory (BILSTM) model; various BERT-based models: model identified as HINBERT, CorefBERT, SpanBERT (based on word segmentation), MTB, CP, and the target natural language processing model obtained in this application embodiment, namely TRUST-BERT; and various RoBERTa-based models: RoBERTa and the target natural language processing model obtained in this application embodiment, namely TRUST-RoBERTa.

[0096] In Table 2, 1%, 10%, 50%, and 100% refer to the number of input samples, respectively. "Masked" refers to the text after masking, and "Standard" refers to the standard text without any processing. Furthermore, Table 2 shows the processing results of traditional models (identified as Fast QA) and models utilizing bi-directional attention flow for machine comprehension (BiDAF), as well as the processing results of various BERT-based models: HINBERT, CorefBERT, SpanBERT, MTB, CP, and the target natural language processing model obtained in this application's embodiments, namely TRUST-BERT, and the processing results of various RoBERTa-based models: RoBERTa and the target natural language processing model obtained in this application's embodiments, namely TRUST-RoBERTa.

[0097] In Table 3, Macro and Micro are two different categories. Furthermore, Table 3 shows the processing results of various BERT-based models: BERT, MTB, CP, the Enhanced Representation Through Knowledge Integration (ERNIE) model, and the target natural language processing model obtained in this application's embodiments, namely TRUST-BERT. It also shows the processing results of various RoBERTa-based models: RoBERTa and the target natural language processing model obtained in this application's embodiments, namely TRUST-RoBERTa.

[0098] Table 1

[0099] Table 2

[0100] Table 3

[0101] As can be seen from Tables 1 to 3, the target natural language processing model described in the embodiments of this application has good performance in the above three different natural language processing tasks, that is, the accuracy is high.

[0102] It should be noted that the entire model acquisition method described in this application embodiment can also be called a training framework. This framework is applicable to various basic processing models, meaning it has good universality. Furthermore, even with insufficient data (e.g., 10%), the target natural language processing model can still reliably complete the processing. Thus, it can be seen that the target natural language processing model obtained in this application embodiment performs well even with insufficient resources, which is very significant for many scenarios with low data annotation capabilities.

[0103] It should also be noted that the order of the steps in the method for obtaining the natural language processing model provided in the embodiments of the present invention can be appropriately adjusted. For example, steps 401 and 402 can be executed simultaneously. Any variations that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the protection scope of this application, and therefore will not be elaborated further.

[0104] In summary, this application provides a method for obtaining a natural language processing model. This method can determine the entity prediction loss value and entity relationship prediction loss value of the basic processing model based on the obtained training samples, and can update and optimize the basic processing model based on these two loss values ​​to obtain the target natural language processing model. Thus, the final target natural language processing model not only has a good understanding of the entities themselves, but also a good understanding of the complex interaction relationships between different entities; that is, the target natural language processing model obtained by this application has good performance.

[0105] Figure 7 This is a structural block diagram of a natural language processing device provided in an embodiment of this application. Figure 7 As shown, the device may include: The text acquisition module 701 can be used to acquire target text.

[0106] The target text may include at least one entity.

[0107] The model acquisition module 702 can be used to acquire the target natural language processing model.

[0108] The target natural language processing model is a model obtained by updating the basic processing model based on the entity prediction loss value and the entity relationship prediction loss value of the basic processing model. The entity prediction loss value is determined based on the similarity of each entity predicted by the basic processing model, and the entity relationship prediction loss value is determined based on the relationship similarity of each entity pair predicted by the basic processing model. Each entity pair includes two entities.

[0109] Processing model 703 can be used to input target text into a target natural language processing model and obtain the processing result output by the target natural language processing model.

[0110] In summary, this application provides a natural language processing apparatus that can input acquired target text into a target natural language processing model to obtain natural language processing results. Since this target natural language processing model is obtained by optimizing the basic processing model based on the entity prediction loss value and entity relationship prediction loss value, it not only has a better understanding of the entities themselves but also a better understanding of the complex interaction relationships between different entities. Therefore, the results of natural language processing using this target natural language processing model have higher accuracy and better reliability.

[0111] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0112] Figure 8 This is a structural block diagram of a natural language processing model acquisition device provided in an embodiment of this application. Figure 8 As shown, the device may include: The sample acquisition module 801 can be used to acquire multiple first training samples and multiple second training samples.

[0113] Each first training sample may include one entity pair, and each second training sample may include two entity pairs. An entity pair may include two entities.

[0114] The first input module 802 can be used to input multiple first training samples into the basic processing model to obtain the entity prediction loss value of the basic processing model. The entity prediction loss value is determined based on the similarity between two entities in each first training sample predicted by the basic processing model.

[0115] The second input module 803 can be used to input multiple second training samples into the basic processing model to obtain the entity relationship prediction loss value of the basic processing model. The entity relationship prediction loss value is determined based on the relationship similarity between two entity pairs in each second training sample predicted by the basic processing model.

[0116] The update module 804 can be used to update the parameters of the basic processing model based on the entity prediction loss value and the entity relationship prediction loss value, so as to obtain the target natural language processing model.

[0117] Optionally, the multiple first training samples may include: M first positive training samples obtained from multiple first documents included in the first document set, and multiple first negative training samples corresponding to each first positive training sample. Each first positive training sample includes: a head entity in a first document and a tail entity of the head entity; each first negative training sample includes: a head entity in the first positive training sample corresponding to the first negative training sample, and a reference entity in the first document to which the head entity belongs. The reference entity is an entity in the first document excluding the tail entity, and M is an integer greater than 1.

[0118] The multiple second training samples may include: N second positive training samples obtained from multiple second documents included in the second document set, and multiple second negative training samples corresponding to each second positive training sample. Each second positive training sample includes: two entity pairs with the same relationship obtained from at least one second document; each second negative training sample includes: two entity pairs with different relationships obtained from at least one second document, and two entity pairs without a relationship; and one entity pair in each second negative training sample is identical to one entity pair in a corresponding second positive training sample, where N is an integer greater than 1.

[0119] Optionally, the first input module 802 can be used to: input each first positive training sample and the corresponding multiple first negative training samples into the basic processing model to obtain the entity prediction loss value of the basic processing model.

[0120] Optionally, the second input module 803 can be used to: input each second positive training sample and the corresponding multiple second negative training samples into the basic processing model to obtain the entity relationship prediction loss value of the basic processing model.

[0121] Optionally, the entity prediction loss value Lep of the basic processing model obtained based on the m-th first positive training sample and the corresponding multiple first negative training samples can satisfy: .

[0122] in, It refers to the similarity between the head entity and the tail entity in the m-th first positive training sample. I refers to the similarity between the head entity and the reference entity included in the j-th first negative training sample among the multiple first negative training samples corresponding to the m-th first positive training sample. I refers to the number of multiple first negative training samples, where m is an integer less than or equal to M and j is an integer less than or equal to I.

[0123] The entity relationship prediction loss value Lrp of the basic processing model obtained based on the nth second positive training sample and the corresponding multiple second negative training samples satisfies:

[0124] in, It refers to the relational similarity between two entity pairs in the nth second positive training sample. Nneg refers to the relational similarity between two entity pairs included in the kth second negative training sample among multiple second negative training samples corresponding to the nth second positive training sample. Nneg refers to the number of multiple second negative training samples, where n is an integer less than or equal to N and k is an integer less than or equal to Nneg.

[0125] Optionally, the update module 804 can be used to update the basic processing model based on the sum of the target loss value, the entity prediction loss value, and the entity relationship prediction loss value, until the update termination condition is met.

[0126] The base processing model that meets the update termination condition is determined as the target natural language processing model.

[0127] The target loss value is used to indicate the computational loss of the underlying processing model itself.

[0128] In summary, this application provides a device for acquiring a natural language processing model. This device can determine the entity prediction loss value and entity relationship prediction loss value of the basic processing model based on the acquired training samples, and can update and optimize the basic processing model based on these two loss values ​​to obtain a target natural language processing model. Thus, the final target natural language processing model not only has a good understanding of the entities themselves, but also a good understanding of the complex interaction relationships between different entities; that is, the target natural language processing model acquired in this application has good performance.

[0129] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0130] Optionally, embodiments of this application provide a computer device, which may include a processor and a memory. The memory may store at least one line of program code, which may be loaded and executed by the processor to achieve, for example... Figure 2 The natural language processing methods shown, or implementations such as Figure 3 or Figure 4 The method for obtaining the natural language processing model is shown.

[0131] Optionally, embodiments of this application provide a computer-readable storage medium storing at least one line of program code, which is loaded and executed by a processor to achieve the following: Figure 2The natural language processing methods shown, or implementations such as Figure 3 or Figure 4 The method for obtaining the natural language processing model is shown.

[0132] Figure 9 This is a structural block diagram of a computer device 900 provided in an embodiment of this application. The computer device can be... Figure 1 The desktop computer shown. Typically, computer device 900 may include: processor 901 and memory 902.

[0133] Processor 901 may include one or more processing cores, such as a quad-core processor, a nine-core processor, etc. Processor 901 may be implemented using at least one hardware form selected from digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). Processor 901 may also include a main processor and a coprocessor. The main processor, also known as the central processing unit (CPU), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 901 may integrate a graphics processing unit (GPU), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 901 may also include an artificial intelligence (AI) processor, which is used to handle computational operations related to machine learning.

[0134] The memory 902 may include one or more computer-readable storage media, which may be non-transitory. The memory 902 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 902 are used to store at least one instruction, which is executed by the processor 901 to implement the various method embodiments described above.

[0135] In some embodiments, the device 900 may also optionally include a peripheral device interface 903 and at least one peripheral device. The processor 901, memory 902, and peripheral device interface 903 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 903 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of the following: a radio frequency circuit 904, a display screen 905, a camera assembly 906, an audio circuit 907, and a power supply 909.

[0136] Peripheral device interface 903 can be used to connect at least one input / output (I / O) related peripheral device to processor 901 and memory 902. In some embodiments, processor 901, memory 902 and peripheral device interface 903 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 901, memory 902 and peripheral device interface 903 can be implemented on separate chips or circuit boards, and this application embodiment does not limit this.

[0137] The radio frequency (RF) circuit 904 is used to receive and transmit radio frequency (RF) signals, also known as electromagnetic signals. The RF circuit 904 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 904 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals back into electrical signals. Optionally, the RF circuit 904 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The RF circuit 904 can communicate with other terminals through at least one wireless communication protocol. This wireless communication protocol includes, but is not limited to: the World Wide Web, metropolitan area networks, intranets, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or wireless fidelity (WiFi) networks. In some embodiments, the RF circuit 904 may also include circuitry related to near-field communication (NFC), which is not limited in this application.

[0138] Display screen 905 is used to display a user interface (UI). The UI may include graphics, text, icons, videos, and any combination thereof. When display screen 905 is a touch display screen, it also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to processor 901 for processing. In this case, display screen 905 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there may be one display screen 905, disposed on the front panel of device 900; in other embodiments, there may be at least two display screens 905, disposed on different surfaces of device 900 or in a folded design; in still other embodiments, display screen 905 may be a flexible display screen, disposed on a curved or folded surface of device 900. Furthermore, display screen 905 may be configured as a non-rectangular, irregular shape, i.e., a non-rectangular screen. Display screen 905 may be made of materials such as liquid crystal displays (LCDs) or organic light-emitting diodes (OLEDs).

[0139] The camera assembly 906 is used to acquire images or videos. Optionally, the camera assembly 906 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is located on the front panel of the terminal, and the rear-facing camera is located on the back of the terminal. In some embodiments, there are at least two rear-facing cameras, which are any one of a main camera, a depth-sensing camera, a wide-angle camera, and a telephoto camera, to achieve background blurring by fusion of the main camera and the depth-sensing camera, panoramic shooting by fusion of the main camera and the wide-angle camera, virtual reality (VR) shooting, or other fusion shooting functions. In some embodiments, the camera assembly 906 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm-light flash and a cool-light flash, which can be used for light compensation at different color temperatures.

[0140] The audio circuit 907 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, converting the sound waves into electrical signals that are input to the processor 901 for processing, or input to the radio frequency circuit 904 for voice communication. For stereo sound acquisition or noise reduction purposes, multiple microphones may be used, each located at a different part of the device 900. The microphone may also be an array microphone or an omnidirectional microphone. The speaker is used to convert the electrical signals from the processor 901 or the radio frequency circuit 904 into sound waves. The speaker may be a conventional diaphragm speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can convert electrical signals not only into audible sound waves but also into inaudible sound waves for purposes such as distance measurement. In some embodiments, the audio circuit 907 may also include a headphone jack.

[0141] Power supply 909 is used to supply power to the various components in device 900. Power supply 909 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 909 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged via a wired line, and a wireless rechargeable battery is a battery that is charged via a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0142] Those skilled in the art will understand that Figure 9 The structure shown does not constitute a limitation on the computer device 900, and may include more or fewer components than shown, or combine certain components, or employ different component arrangements.

[0143] In the embodiments of this application, the terms "first" and "second" are used to distinguish identical or similar items with essentially the same function. It should be understood that there is no logical or temporal dependency between "first", "second", and "nth", nor is there any limitation on the quantity or execution order.

[0144] In the embodiments of this application, the term "at least one" means one or more, and the term "multiple" means two or more.

[0145] The term "and / or" as used in the embodiments of this application refers to and covers any and all possible combinations of one or more of the associated listed items. The term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Additionally, the character " / " in this application generally indicates that the preceding and following related objects have an "or" relationship.

[0146] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A natural language processing method, characterized in that, The method includes: Obtain target text, which includes at least one entity; obtain a target natural language processing model; input the target text into the target natural language processing model to obtain the processing result output by the target natural language processing model; The target natural language processing model is trained in the following manner: Obtain multiple first training samples and multiple second training samples; Each of the first training samples includes one entity pair, and each of the second training samples includes two entity pairs, the entity pairs including a head entity and a tail entity; the first training samples are samples used to train the basic processing model to predict entities, and the first training samples are extracted from each of the first documents. The plurality of first training samples include a plurality of first positive training samples and a plurality of first negative training samples corresponding to each first positive training sample. Each first positive training sample includes a head entity and a tail entity of the head entity in a first document. Each first negative training sample includes a head entity in the first positive training sample corresponding to the first negative training sample and a reference entity in the first document to which the head entity belongs. The reference entity is an entity in the first document other than the tail entity. The second training sample is a sample used to train the basic processing model to predict entity relationships. The second training sample is extracted from a second document other than the first document. The plurality of second training samples include a plurality of second positive training samples and a plurality of second negative training samples corresponding to each second positive training sample. Each second positive training sample includes two entity pairs with the same relationship obtained from at least one second document. Each second negative training sample includes two entity pairs with different relationships and two entity pairs with no relationship obtained from at least one second document. Each first positive training sample and its corresponding multiple first negative training samples are sequentially input into the basic processing model. Each input yields an entity prediction loss value for the basic processing model. The entity prediction loss value is determined based on the similarity between two entities in each of the first training samples predicted by the basic processing model. Each second positive training sample and its corresponding multiple second negative training samples are sequentially input into the basic processing model. Each input yields an entity relationship prediction loss value for the basic processing model. The entity relationship prediction loss value is determined based on the relationship similarity between two entity pairs in each of the second training samples predicted by the basic processing model. Each time a training sample is input and an entity prediction loss value and an entity relationship prediction loss value are obtained, the parameters of the basic processing model are updated once based on the entity prediction loss value and the entity relationship prediction loss value, until the target natural language processing model is obtained.

2. A method for obtaining a natural language processing model, characterized in that, The method includes: Obtain multiple first training samples and multiple second training samples; Each first training sample includes one entity pair, and each second training sample includes two entity pairs, each entity pair including a head entity and a tail entity. The first training samples are used to train the basic processing model for entity prediction. The first training samples are extracted from each first document. The plurality of first training samples include a plurality of first positive training samples and a plurality of first negative training samples corresponding to each first positive training sample. Each first positive training sample includes a head entity and a tail entity of the head entity in a first document. Each first negative training sample includes the head entity in the first positive training sample corresponding to the first negative training sample and the tail entity of the head entity. The reference entity in the first document is an entity other than the tail entity in the first document; the second training sample is a sample used to train the basic processing model to predict entity relationships. The second training sample is extracted from a second document other than the first document. The plurality of second training samples include a plurality of second positive training samples and a plurality of second negative training samples corresponding to each second positive training sample. Each second positive training sample includes two entity pairs with the same relationship obtained from at least one second document. Each second negative training sample includes two entity pairs with different relationships and two entity pairs with no relationship obtained from at least one second document. Each first positive training sample and its corresponding multiple first negative training samples are sequentially input into the basic processing model. Each input yields an entity prediction loss value for the basic processing model. The entity prediction loss value is determined based on the similarity between two entities in each of the first training samples predicted by the basic processing model. Each second positive training sample and its corresponding multiple second negative training samples are sequentially input into the basic processing model. Each input yields an entity relationship prediction loss value for the basic processing model. The entity relationship prediction loss value is determined based on the relationship similarity between two entity pairs in each of the second training samples predicted by the basic processing model. Each time a training sample is input and an entity prediction loss value and an entity relationship prediction loss value are obtained, the parameters of the basic processing model are updated once based on the entity prediction loss value and the entity relationship prediction loss value, until the target natural language processing model is obtained.

3. The method according to claim 2, characterized in that, The plurality of first training samples include: M first positive training samples obtained from a plurality of first documents included in the first document set, and a plurality of first negative training samples corresponding to each first positive training sample; M is an integer greater than 1; The plurality of second training samples include: N second positive training samples obtained from a plurality of second documents included in the second document set, and a plurality of second negative training samples corresponding to each second positive training sample; N is an integer greater than 1.

4. The method according to claim 2, characterized in that, The entity prediction loss value Lep of the basic processing model obtained based on the m-th first positive training sample and the corresponding multiple first negative training samples satisfies: ； in, It refers to the similarity between the head entity and the tail entity in the m-th first positive training sample. It refers to the similarity between the head entity and the reference entity included in the j-th first negative training sample among the multiple first negative training samples corresponding to the m-th first positive training sample, where I refers to the number of the multiple first negative training samples, and m is an integer less than or equal to M, and j is an integer less than or equal to I. The entity relationship prediction loss value Lrp of the basic processing model obtained based on the nth second positive training sample and the corresponding multiple second negative training samples satisfies: ； in, It refers to the relational similarity between two entity pairs in the nth second positive training sample. Nneg refers to the relational similarity between two entity pairs included in the kth second negative training sample among the multiple second negative training samples corresponding to the nth second positive training sample. Nneg refers to the number of the multiple second negative training samples, where n is an integer less than or equal to N and k is an integer less than or equal to Nneg.

5. The method according to any one of claims 2 to 4, characterized in that, After each input training sample yields an entity prediction loss value and an entity relationship prediction loss value, the parameters of the basic processing model are updated based on these two values ​​until the target natural language processing model is obtained. This process includes: The basic processing model is updated based on the sum of the target loss value, the entity prediction loss value, and the entity relationship prediction loss value until the update termination condition is met. The base processing model that meets the update termination condition is determined as the target natural language processing model; The target loss value is used to indicate the computational loss of the basic processing model itself.

6. A natural language processing device, characterized in that, The device includes: A text acquisition module is used to acquire target text, wherein the target text includes at least one entity; The model acquisition module is used to acquire the target natural language processing model; A processing model is used to input the target text into the target natural language processing model and obtain the processing result output by the target natural language processing model; The target natural language processing model is trained in the following manner: Obtain multiple first training samples and multiple second training samples; Each first training sample includes one entity pair, and each second training sample includes two entity pairs, each entity pair including a head entity and a tail entity. The first training samples are used to train the basic processing model for entity prediction. The first training samples are extracted from each first document. The plurality of first training samples include a plurality of first positive training samples and a plurality of first negative training samples corresponding to each first positive training sample. Each first positive training sample includes a head entity and a tail entity of the head entity in a first document. Each first negative training sample includes the head entity in the first positive training sample corresponding to the first negative training sample and the tail entity of the head entity. The reference entity in the first document is an entity other than the tail entity in the first document; the second training sample is a sample used to train the basic processing model to predict entity relationships. The second training sample is extracted from a second document other than the first document. The plurality of second training samples include a plurality of second positive training samples and a plurality of second negative training samples corresponding to each second positive training sample. Each second positive training sample includes two entity pairs with the same relationship obtained from at least one second document. Each second negative training sample includes two entity pairs with different relationships and two entity pairs with no relationship obtained from at least one second document. Each first positive training sample and its corresponding multiple first negative training samples are sequentially input into the basic processing model. Each input yields an entity prediction loss value for the basic processing model. The entity prediction loss value is determined based on the similarity between two entities in each of the first training samples predicted by the basic processing model. Each second positive training sample and its corresponding multiple second negative training samples are sequentially input into the basic processing model. Each input yields an entity relationship prediction loss value for the basic processing model. The entity relationship prediction loss value is determined based on the relationship similarity between two entity pairs in each of the second training samples predicted by the basic processing model. Each time a training sample is input and an entity prediction loss value and an entity relationship prediction loss value are obtained, the parameters of the basic processing model are updated once based on the entity prediction loss value and the entity relationship prediction loss value, until the target natural language processing model is obtained.

7. A device for acquiring a natural language processing model, characterized in that, The device includes: The sample acquisition module is used to acquire multiple first training samples and multiple second training samples; Each first training sample includes one entity pair, and each second training sample includes two entity pairs, each entity pair including a head entity and a tail entity. The first training samples are used to train the basic processing model for entity prediction. The first training samples are extracted from each first document. The plurality of first training samples include a plurality of first positive training samples and a plurality of first negative training samples corresponding to each first positive training sample. Each first positive training sample includes a head entity and a tail entity of the head entity in a first document. Each first negative training sample includes the head entity in the first positive training sample corresponding to the first negative training sample and the tail entity of the head entity. The reference entity in the first document is an entity other than the tail entity in the first document; the second training sample is a sample used to train the basic processing model to predict entity relationships. The second training sample is extracted from a second document other than the first document. The plurality of second training samples include a plurality of second positive training samples and a plurality of second negative training samples corresponding to each second positive training sample. Each second positive training sample includes two entity pairs with the same relationship obtained from at least one second document. Each second negative training sample includes two entity pairs with different relationships and two entity pairs with no relationship obtained from at least one second document. The first input model is used to sequentially input each first positive training sample and the corresponding multiple first negative training samples into the basic processing model. Each input yields an entity prediction loss value for the basic processing model. The entity prediction loss value is determined based on the similarity between two entities in each of the first training samples predicted by the basic processing model. The second input model is used to sequentially input each second positive training sample and the corresponding multiple second negative training samples into the basic processing model. Each input yields an entity relationship prediction loss value for the basic processing model. The entity relationship prediction loss value is determined based on the relationship similarity between two entity pairs in each of the second training samples predicted by the basic processing model. The update module is used to update the parameters of the basic processing model once each time an entity prediction loss value and an entity relationship prediction loss value are obtained from input training samples, until the target natural language processing model is obtained.

8. The apparatus according to claim 7, characterized in that, The plurality of first training samples include: M first positive training samples obtained from a plurality of first documents included in the first document set, and a plurality of first negative training samples corresponding to each first positive training sample; M is an integer greater than 1; The plurality of second training samples include: N second positive training samples obtained from a plurality of second documents included in the second document set, and a plurality of second negative training samples corresponding to each second positive training sample; N is an integer greater than 1.

9. The apparatus according to claim 7, characterized in that, The entity prediction loss value Lep of the basic processing model obtained based on the m-th first positive training sample and the corresponding multiple first negative training samples satisfies: ； in, It refers to the similarity between the head entity and the tail entity in the m-th first positive training sample. It refers to the similarity between the head entity and the reference entity included in the j-th first negative training sample among the multiple first negative training samples corresponding to the m-th first positive training sample, where I refers to the number of the multiple first negative training samples, and m is an integer less than or equal to M, and j is an integer less than or equal to I. The entity relationship prediction loss value Lrp of the basic processing model obtained based on the nth second positive training sample and the corresponding multiple second negative training samples satisfies: ； in, It refers to the relational similarity between two entity pairs in the nth second positive training sample. Nneg refers to the relational similarity between two entity pairs included in the kth second negative training sample among the multiple second negative training samples corresponding to the nth second positive training sample. Nneg refers to the number of the multiple second negative training samples, where n is an integer less than or equal to N and k is an integer less than or equal to Nneg.

10. The apparatus according to any one of claims 7 to 9, characterized in that, The update module is used for: The basic processing model is updated based on the sum of the target loss value, the entity prediction loss value, and the entity relationship prediction loss value until the update termination condition is met. The base processing model that meets the update termination condition is determined as the target natural language processing model; The target loss value is used to indicate the computational loss of the basic processing model itself.

11. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one piece of program code, the at least one piece of program code being loaded and executed by the processor to implement the natural language processing method as described in claim 1, or to implement the method for obtaining a natural language processing model as described in any one of claims 2 to 5.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one piece of program code, which is loaded and executed by a processor to implement the natural language processing method as described in claim 1, or to implement the method for obtaining a natural language processing model as described in any one of claims 2 to 5.

Images