Large language model training method, device, electronic device and storage medium

Abstract

The present disclosure proposes a training method, device, electronic device and storage medium for a large language model, which relates to the field of artificial intelligence technology, especially to the field of deep learning, deep neural network and large language model technology. The method includes: obtaining a sample text containing logical reasoning content of a target reasoning method; obtaining a credibility evaluation result of the sample text; correlating the sample text and the credibility evaluation result of the sample text to obtain a training sample; obtaining a training sample set based on multiple training samples, and training the initial model based on the training sample set to obtain a large language model. Thus, the training of the large language model can be achieved based on the sample text containing logical reasoning content of the target reasoning method and the credibility evaluation result of the sample text, so that the large language model can learn the target reasoning method during the training process, which helps to improve the text credibility evaluation effect of the large language model, and is particularly suitable for application scenarios of news credibility evaluation.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, in particular to the fields of deep learning, deep neural networks, and large language model technology, and in particular to a large language model training method, a news credibility evaluation method, a device, an electronic device, a storage medium, and a computer program product. Background Art

[0002] With the continuous development of artificial intelligence (AI), large language models (LLMs) have gained widespread application in fields such as information extraction, text credibility assessment, and machine translation due to their advantages such as good generalization. However, existing methods for training LLMs in related technologies suffer from poor text credibility assessment results using trained LLMs. Summary of the Invention

[0003] The present disclosure proposes a large language model training method, a news credibility evaluation method, a device, an electronic device, a storage medium, and a computer program product.

[0004] According to a first aspect of the present disclosure, a method for training a large language model is proposed, comprising: obtaining a sample text containing logical reasoning content of a target reasoning method; obtaining a credibility assessment result of the sample text; associating the sample text with the credibility assessment result of the sample text to obtain a training sample; obtaining a training sample set based on a plurality of the training samples, and training an initial model based on the training sample set to obtain a large language model.

[0005] According to a second aspect of the present disclosure, a method for evaluating the credibility of news is proposed, comprising: obtaining target news; inputting the target news into a large language model, performing credibility evaluation on the target news through the large language model, and obtaining a credibility evaluation result of the target news, wherein the large language model is obtained using the large language model training method proposed in the first aspect above.

[0006] According to a third aspect of the present disclosure, a training device for a large language model is proposed, comprising: a first acquisition module for acquiring a sample text containing logical reasoning content of a target reasoning method; a second acquisition module for acquiring a credibility assessment result of the sample text; an association module for associating the sample text with the credibility assessment result of the sample text to obtain a training sample; and a training module for obtaining a training sample set based on a plurality of the training samples, and training an initial model based on the training sample set to obtain a large language model.

[0007] According to a fourth aspect of the present disclosure, a device for evaluating the credibility of news is proposed, comprising: an acquisition module for acquiring target news; an evaluation module for inputting the target news into a large language model, performing credibility evaluation on the target news through the large language model, and obtaining a credibility evaluation result of the target news, wherein the large language model is obtained using the large language model training method proposed in the first aspect above.

[0008] According to a fifth aspect of the present disclosure, an electronic device is proposed, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the large language model training method proposed in the first aspect above, or to execute the news credibility evaluation method proposed in the second aspect above.

[0009] According to the sixth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is proposed, wherein the computer instructions are used to enable the computer to execute the large language model training method proposed in the first aspect, or to execute the news credibility evaluation method proposed in the second aspect.

[0010] According to the seventh aspect of the present disclosure, a computer program product is proposed, including a computer program, which, when executed by a processor, implements the large language model training method proposed in the first aspect, or implements the news credibility evaluation method proposed in the second aspect.

[0011] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0013] Figure 1 Schematic diagram of a flow chart of a method for training a large language model according to an embodiment of the present disclosure;

[0014] Figure 2 A flowchart of a method for training a large language model according to another embodiment of the present disclosure is provided;

[0015] Figure 3 A flowchart of a method for training a large language model according to another embodiment of the present disclosure is provided;

[0016] Figure 4A flowchart of a method for training a large language model according to another embodiment of the present disclosure is provided;

[0017] Figure 5 A flowchart of a method for training a large language model according to another embodiment of the present disclosure is provided;

[0018] Figure 6 Schematic diagram of a flow chart of a method for evaluating news credibility according to an embodiment of the present disclosure;

[0019] Figure 7 This is a schematic diagram of the structure of a large language model training device according to an embodiment of the present disclosure;

[0020] Figure 8 This is a structural diagram of a news credibility evaluation device according to an embodiment of the present disclosure;

[0021] Figure 9 A schematic block diagram of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0022] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0023] Artificial Intelligence (AI) is a discipline that studies and develops theories, methods, technologies, and application systems for simulating, extending, and expanding human intelligence. Currently, AI technology has been widely used due to its high degree of automation, high precision, and low cost.

[0024] DL (Deep Learning) is a new research direction in the field of ML (Machine Learning). It is a science that learns the inherent laws and representation levels of sample data, enabling machines to have analytical and learning capabilities like humans and recognize data such as text, images, and sounds. It is widely used in speech and image recognition.

[0025] A Deep Neural Network (DNN) is a technology in the field of Machine Learning (ML). It is a neural network with at least one hidden layer. To overcome the shortcomings of traditional neural networks, such as overfitting and slow training speed, DNNs generally use a layer-by-layer pre-training mechanism, rather than the traditional backpropagation training mechanism.

[0026] Large Language Models (LLMs) are deep learning models trained using large amounts of text data. They can generate natural language text or understand the meaning of text. Large language models can handle a variety of natural language tasks, such as text classification, question-answering, and conversation, and are a key path to artificial intelligence.

[0027] Figure 1 FIG. 1 is a flow chart of a method for training a large language model according to an embodiment of the present disclosure. Figure 1 As shown, the method includes:

[0028] S101, obtaining a sample text containing logical reasoning content of a target reasoning method.

[0029] It should be noted that the execution entity of the large language model training method of the embodiments of the present disclosure may be a hardware device with data information processing capabilities and / or the necessary software to drive the operation of the hardware device. Optionally, the execution entity may include a workstation, server, computer, user terminal, and other intelligent devices. Among them, user terminals include but are not limited to mobile phones, computers, intelligent voice interaction devices, smart appliances, and in-vehicle terminals.

[0030] It should be noted that sample text refers to text that contains logical reasoning content that follows the target reasoning method. There are no specific restrictions on sample text; for example, sample text can include text in at least one language, such as Chinese or English. For example, sample text can include news, essays, commentaries, and literary works.

[0031] It should be noted that there are no specific restrictions on target reasoning methods. For example, they may include hypothetical reasoning, analogical reasoning, inductive reasoning, and critical reasoning. Critical reasoning may include logical reasoning, causal analysis, and evidence evaluation. The number of target reasoning methods is at least one.

[0032] It should be noted that there are not too many restrictions on the content of logical reasoning. For example, it can include arguments, evidence, reasoning, premises, conclusions, etc.

[0033] In one embodiment, obtaining a sample text containing logical reasoning content for a target reasoning method includes identifying candidate reasoning methods for a set text in a set text library. If the candidate reasoning methods include at least one target reasoning method, the set text is determined as the sample text. It should be noted that identifying candidate reasoning methods for a set text in the set text library can be described in the following embodiments and will not be further described here.

[0034] It should be noted that the setting text library includes multiple setting texts, which can be pre-set.

[0035] For example, if the set text library includes set texts A, B, and C, the target reasoning method includes critical reasoning, the candidate reasoning methods for identifying the set text A include hypothetical reasoning and critical reasoning, the candidate reasoning methods for identifying the set text B include deductive reasoning, and the candidate reasoning methods for identifying the set text C include analogical reasoning, then the candidate reasoning methods for the set text A include the target reasoning method, and the candidate reasoning methods for the set texts B and C do not include the target reasoning method, and the set text A is determined as the sample text.

[0036] In one embodiment, obtaining a sample text containing logical reasoning content of a target reasoning method includes obtaining a label of a setting text in a setting text library. If the label of the setting text indicates that the setting text contains logical reasoning content of the target reasoning method, the setting text is determined as a sample text.

[0037] It should be noted that there are not too many restrictions on the label of the setting text. For example, if the label of the setting text is 1, it indicates that the setting text contains the logical reasoning content of the target reasoning method. If the label of the setting text is 0, it indicates that the setting text does not contain the logical reasoning content of the target reasoning method.

[0038] In some examples, the label of the set text may be manually annotated, or the set text may be input into a first annotation model, and the first annotation model may output the label of the set text.

[0039] S102: Obtain the credibility evaluation result of the sample text.

[0040] It should be noted that there are no excessive restrictions on the credibility assessment results of the sample text. For example, it may include the credibility of the sample text and the reasons for the credibility assessment.

[0041] In one embodiment, the credibility of the sample text ranges from 0 to 100.

[0042] In one embodiment, the evaluation reasons may include evaluation reasons for the true part of the sample text, and may also include evaluation reasons for the false part of the sample text.

[0043] In one embodiment, the credibility evaluation result of the sample text may be annotated manually, or the sample text may be input into a second annotation model, and the second annotation model may output the credibility evaluation result of the sample text.

[0044] In one embodiment, obtaining the credibility evaluation result of the sample text includes querying target knowledge associated with the sample text from a set knowledge base based on the sample text, performing credibility evaluation on the sample text based on the sample text and the target knowledge, and obtaining the credibility evaluation result of the sample text.

[0045] It should be noted that the set knowledge base may include set knowledge in multiple knowledge fields, including science, literature, philosophy, art, etc. Target knowledge refers to the set knowledge associated with the sample text in the set knowledge base.

[0046] In some examples, based on the sample text, querying a predetermined knowledge base for target knowledge associated with the sample text includes determining the knowledge domain to which the logical reasoning content in the sample text belongs, and querying the predetermined knowledge base for target knowledge associated with the sample text based on the knowledge domain corresponding to the sample text. It should be noted that determining the knowledge domain to which the logical reasoning content in the sample text belongs can be described in the following embodiments and will not be further elaborated here.

[0047] For example, if the knowledge domain corresponding to sample text A is medicine, the set knowledge under medicine can be queried from the set knowledge base, and the queried set knowledge under medicine can be used as the target knowledge associated with sample text A.

[0048] For example, if the knowledge domain corresponding to sample text B is geography, the set knowledge under geography can be queried from the set knowledge base, and the queried set knowledge under geography can be used as the target knowledge associated with sample text B.

[0049] In some examples, based on the sample text and the target knowledge, the sample text is credibility evaluated to obtain a credibility evaluation result of the sample text, including identifying whether there is conflicting content between the sample text and the target knowledge, and the sample text is credibility evaluated based on the identification result to obtain a credibility evaluation result of the sample text.

[0050] In some examples, the recognition results include the number of conflicting contents, the difference level of the conflicting contents, etc. A higher difference level indicates a greater difference between the sample content and the target knowledge.

[0051] In some examples, the credibility of the sample text is negatively correlated with the amount of conflicting content, and the credibility of the sample text is negatively correlated with the level of difference of the conflicting content.

[0052] In some examples, a credibility assessment reason for the sample text may be obtained based on the conflicting content.

[0053] For example, if the value of data x in sample text A is 100, the value of data x in sample text B is 110, and the value of data x in target knowledge is 80, that is, there is conflicting content between sample text A and target knowledge, and the conflicting content is the value of data x. The difference level of the conflicting content of sample text A is the first level. There is conflicting content between sample text B and target knowledge, and the conflicting content is the value of data x. The difference level of the conflicting content of sample text B is the second level. The first level is lower than the second level.

[0054] The credibility of the sample text A is 80, and the credibility evaluation reason is "the value of the data x in the sample text A is 100, the value of the data x in the target knowledge is 80, and the difference between the two is 20."

[0055] The credibility of the sample text B is 60, and the credibility assessment reason is "the value of the data x in the sample text B is 110, the value of the data x in the target knowledge is 80, and the difference between the two is 30."

[0056] S103: Associating the sample text with the credibility evaluation result of the sample text to obtain a training sample.

[0057] It should be noted that associating the sample text and the credibility evaluation result of the sample text can be achieved by using any data association method in the relevant technology, and no further limitation is made here.

[0058] In one embodiment, the sample text and the credibility evaluation result of the sample text are associated to obtain the training sample, which includes establishing a mapping relationship, a corresponding relationship, or other association relationship between the sample text and the credibility evaluation result of the sample text.

[0059] For example, a mapping relationship between sample text A and the credibility evaluation result of sample text A can be established to obtain a training sample.

[0060] S104: obtaining a training sample set based on the multiple training samples, and training the initial model based on the training sample set to obtain a large language model.

[0061] It can be understood that the training sample set includes multiple training samples.

[0062] In one embodiment, obtaining the training sample set based on the plurality of training samples includes randomly screening a set number of training samples to obtain the training sample set.

[0063] It should be noted that the initial model is trained based on the training sample set to obtain a large language model, which can be achieved by using any large language model training method in the relevant technology, and no further restrictions are made here.

[0064] In one embodiment, an initial model is trained based on a training sample set to obtain a large language model, including inputting sample text into the initial model, having the initial model output a credibility prediction evaluation result of the sample text, obtaining a loss function of the initial model based on the credibility evaluation result of the sample text and the credibility prediction evaluation result of the sample text, and training the initial model based on the loss function of the initial model.

[0065] The training method for a large language model proposed in the present disclosure obtains a sample text containing logical reasoning content of a target reasoning method, obtains a credibility assessment result of the sample text, associates the sample text with the credibility assessment result of the sample text to obtain a training sample, obtains a training sample set based on multiple training samples, and trains an initial model based on the training sample set to obtain a large language model. Thus, based on the sample text containing logical reasoning content of the target reasoning method and the credibility assessment result of the sample text, a training sample can be obtained to achieve the training of the large language model. Thus, during the training process, the large language model can learn the target reasoning method so that the credibility assessment of the text can be performed subsequently according to the target reasoning method, which helps to improve the text credibility assessment effect of the large language model and is particularly suitable for application scenarios of news credibility assessment.

[0066] In the above embodiment, in step S101, the sample text containing the logical reasoning content of the target reasoning method can be obtained in combination with Figure 2 Further understanding, Figure 2 This is a flow chart of a method for training a large language model according to another embodiment of the present disclosure. Figure 2 As shown, the method includes:

[0067] S201, obtaining target keywords for logical reasoning corresponding to the target reasoning method.

[0068] S202: Determine whether the setting text in the setting text library contains the target keyword.

[0069] S203: If the set text contains the target keyword, the set text is determined as a sample text.

[0070] It is understood that the target keywords corresponding to the target reasoning method can be pre-set. One target reasoning method can correspond to at least one target keyword. Different target reasoning methods can correspond to different target keywords, or even the same target keyword. This is not limited here.

[0071] For example, if the set text library includes set texts A, B, and C, and the target reasoning method includes causal analysis, the target keywords corresponding to the causal analysis may include "because", "so", "reason", "result", "sufficient", "necessary", "condition", etc.

[0072] If the set text A includes the target keywords "because" and "so", the set text A may be determined as the sample text.

[0073] If the set text B contains the target keywords "cause" and "result", the set text B may be determined as the sample text.

[0074] If the set text C does not include any target keyword corresponding to the causal relationship analysis, the set text C is not determined as a sample text.

[0075] S204: Obtain the credibility evaluation result of the sample text.

[0076] S205 , associating the sample text with the credibility evaluation result of the sample text to obtain a training sample.

[0077] S206: Obtain a training sample set based on the multiple training samples, and train the initial model based on the training sample set to obtain a large language model.

[0078] For the relevant contents of steps S204-S206, please refer to the above embodiment and will not be repeated here.

[0079] The large language model training method proposed in this disclosure obtains target keywords for logical reasoning corresponding to the target reasoning method, determines whether a set text in a set text library contains the target keywords, and if so, determines the set text as a sample text. Thus, when the set text contains the target keywords, the set text is determined as a sample text, thereby achieving sample text acquisition.

[0080] In the above embodiment, in step S101, the sample text containing the logical reasoning content of the target reasoning method can be obtained in combination with Figure 3 Further understanding, Figure 3 This is a flow chart of a method for training a large language model according to another embodiment of the present disclosure. Figure 3 As shown, the method includes:

[0081] S301: Identify candidate reasoning methods for a setting text in a setting text library.

[0082] S302: If the candidate reasoning method is consistent with the target reasoning method, the set text is determined as the sample text.

[0083] In one embodiment, identifying candidate inference methods for a set text in a set text library includes performing semantic analysis on the set text to obtain the candidate inference methods. It should be noted that semantic analysis can be implemented using any semantic analysis method in the relevant art, and is not limited here.

[0084] For example, if the set text library includes set texts A, B, and C, the target reasoning method includes critical reasoning, the candidate reasoning method for identifying set text A includes hypothetical reasoning, the candidate reasoning method for identifying set text B includes critical reasoning, and the candidate reasoning method for identifying set text C includes analogical reasoning, then the candidate reasoning method for setting text B is consistent with the target reasoning method, and the candidate reasoning methods for setting texts A and C are inconsistent with the target reasoning method, and the setting text B is determined as the sample text.

[0085] S303: Obtain the credibility evaluation result of the sample text.

[0086] S304: Associating the sample text with the credibility evaluation result of the sample text to obtain a training sample.

[0087] S305: Obtain a training sample set based on the multiple training samples, and train the initial model based on the training sample set to obtain a large language model.

[0088] For the relevant contents of steps S303-S305, please refer to the above embodiment and will not be repeated here.

[0089] The large language model training method proposed in this disclosure identifies candidate inference methods for a set text in a set text library. If the candidate inference method matches the target inference method, the set text is determined as a sample text. Thus, when the candidate inference method matches the target inference method, the set text can be determined as a sample text, thereby achieving sample text acquisition.

[0090] In the above embodiment, in step S104, a training sample set is obtained based on multiple training samples. Figure 4 Further understanding, Figure 4 This is a flow chart of a method for training a large language model according to another embodiment of the present disclosure. Figure 4 As shown, the method includes:

[0091] S401, obtaining a sample text containing logical reasoning content of a target reasoning method.

[0092] S402: Obtain the credibility evaluation result of the sample text.

[0093] S403: Associating the sample text with the credibility evaluation result of the sample text to obtain a training sample.

[0094] For the relevant contents of steps S401-S403, please refer to the above embodiment and will not be repeated here.

[0095] S404 , selecting an i-th first subset from a plurality of training samples under the i-th target reasoning mode.

[0096] S405 , obtaining a union of N first subsets as a training sample set.

[0097] Where N is a positive integer, and i is a positive integer not greater than N. The number of target reasoning methods is N. There is no excessive restriction on N.

[0098] It should be noted that the target reasoning method corresponds one-to-one to the first subset. The i-th first subset includes at least one training sample under the i-th target reasoning method, and the reasoning method of the logical reasoning content of the sample text corresponding to the training sample in the i-th first subset is the i-th target reasoning method.

[0099] In one embodiment, the i-th first subset is selected from multiple training samples under the i-th target reasoning mode, including randomly selecting a set number of training samples from multiple training samples under the i-th target reasoning mode to obtain the i-th first subset.

[0100] For example, the target reasoning methods include logical reasoning and causal relationship analysis. The first subset A can be screened out from multiple training samples under logical reasoning, and the first subset B can be screened out from multiple training samples under causal relationship analysis. The union of the first subset A and the first subset B is obtained as the training sample set.

[0101] S406: Train the initial model based on the training sample set to obtain a large language model.

[0102] For the relevant content of step S406, please refer to the above embodiment and will not be repeated here.

[0103] The large language model training method proposed in this disclosure selects the i-th first subset from multiple training samples under the i-th target reasoning method, and obtains the union of N first subsets as the training sample set. This method comprehensively considers training samples under multiple target reasoning methods to obtain a training sample set to train the large language model. As a result, the large language model can learn multiple target reasoning methods during training, which helps improve the text credibility assessment and generalization performance of the large language model.

[0104] In the above embodiment, in step S104, a training sample set is obtained based on multiple training samples. Figure 5 Further understanding, Figure 5 This is a flow chart of a method for training a large language model according to another embodiment of the present disclosure. Figure 5 As shown, the method includes:

[0105] S501, obtaining a sample text containing logical reasoning content of a target reasoning method.

[0106] S502: Obtain the credibility evaluation result of the sample text.

[0107] S503: Associating the sample text with the credibility evaluation result of the sample text to obtain a training sample.

[0108] For the relevant contents of steps S501-S503, please refer to the above embodiment and will not be repeated here.

[0109] S504: Determine the knowledge domain to which the logical reasoning content in the sample text belongs, and use this as the knowledge domain to which the training sample corresponding to the sample text belongs.

[0110] In one embodiment, determining the knowledge domain to which the logical reasoning content in the sample text belongs includes performing semantic analysis on the logical reasoning content in the sample text to obtain the knowledge domain to which the logical reasoning content in the sample text belongs.

[0111] For example, if the logical reasoning content in sample text A includes "heatstroke" and "body temperature", it can be determined that the knowledge field to which the logical reasoning content in the sample text belongs is medicine, and medicine is used as the knowledge field to which the training sample corresponding to sample text A belongs.

[0112] For example, if the logical reasoning content in sample text B includes "typhoon" and "heavy rain", it can be determined that the knowledge field to which the logical reasoning content in the sample text belongs is meteorology, and meteorology is used as the knowledge field to which the training sample corresponding to sample text B belongs.

[0113] S505 , based on the knowledge domain to which the training samples belong, a training sample set is screened out from the plurality of training samples.

[0114] In one embodiment, based on the knowledge domain to which the training samples belong, a training sample set is screened from a plurality of training samples, including screening a set number of training samples in a knowledge domain from the plurality of training samples to obtain the training sample set.

[0115] In one embodiment, a training sample set is selected from multiple training samples based on the knowledge domain to which the training samples belong. This includes obtaining M target knowledge domains, selecting the jth second subset from multiple training samples under the jth target knowledge domain, and obtaining the union of the M second subsets as the training sample set. Thus, by comprehensively considering the training samples under multiple target knowledge domains, a training sample set is obtained to train a large language model, making the large language model applicable to multiple target knowledge domains, thereby improving the text credibility assessment and generalization performance of the large language model.

[0116] Where M is a positive integer, and j is a positive integer not greater than M. The number of target knowledge domains is M. There is no excessive restriction on M. There is no excessive restriction on the target knowledge domains.

[0117] It should be noted that the target knowledge domain corresponds to the second subset one-to-one, the jth second subset includes at least one training sample under the jth target knowledge domain, and the knowledge domain to which the training sample in the jth second subset belongs is the jth target knowledge domain.

[0118] In one embodiment, selecting the jth second subset from multiple training samples under the jth target knowledge domain includes randomly selecting a set number of training samples from the multiple training samples under the jth target knowledge domain to obtain the jth second subset.

[0119] For example, the target knowledge domain includes medicine and meteorology. The second subset A can be screened out from multiple training samples under medicine, and the second subset B can be screened out from multiple training samples under meteorology. The union of the second subset A and the second subset B is obtained as the training sample set.

[0120] S506: Train the initial model based on the training sample set to obtain a large language model.

[0121] For the relevant content of step S506, please refer to the above embodiment and will not be repeated here.

[0122] The large language model training method proposed in this disclosure determines the knowledge domain to which the logical reasoning content in a sample text belongs, uses this as the knowledge domain to which the training sample corresponding to the sample text belongs, and then selects a training sample set from multiple training samples based on the knowledge domain to which the training sample belongs. This allows the training sample set to be selected from multiple training samples, taking into account the knowledge domain to which the training sample belongs, thereby increasing the flexibility of selecting the training sample set.

[0123] Figure 6 FIG. 1 is a flow chart of a method for evaluating news credibility according to an embodiment of the present disclosure. Figure 6 As shown, the method includes:

[0124] S601, obtaining target news.

[0125] It should be noted that the execution entity of the news credibility assessment method of the disclosed embodiments may be a hardware device with data processing capabilities and / or the necessary software to drive the operation of the hardware device. Alternatively, the execution entity may include a workstation, server, computer, user terminal, and other intelligent device. User terminals include, but are not limited to, mobile phones, computers, intelligent voice interaction devices, smart home appliances, and in-vehicle terminals.

[0126] It should be noted that there are no excessive restrictions on target news. For example, target news includes but is not limited to newspapers, news apps (applications), and news web pages.

[0127] In one embodiment, taking the execution subject as a user terminal as an example, the user terminal can obtain the target news from its own storage space and / or crawl the target news from a web page or an APP (Application).

[0128] In one embodiment, taking the execution subject as a server as an example, obtaining target news may include receiving the target news sent by a client. It is understood that the client may obtain the target news based on the operation information of the user operating the client (e.g., the user uploading the target news, the user clicking the target news icon on the client display interface, etc.), and send the target news to the server. Correspondingly, the server may receive the target news sent by the client.

[0129] S602: Input the target news into the large language model, perform credibility evaluation on the target news through the large language model, and obtain a credibility evaluation result of the target news, wherein the large language model is obtained by using a large language model training method.

[0130] It should be noted that large language models can be used Figures 1 to 5 The training method of the large language model shown is obtained and will not be repeated here.

[0131] In one embodiment, a credibility assessment of target news is performed using a large language model, and a credibility assessment result for the target news is obtained. This includes extracting information from the target news using the large language model to obtain key information, querying reference knowledge for the key information from a predetermined knowledge base based on the semantic representation of the key information using the large language model, and performing a credibility assessment of the target news using the large language model based on the key information and the reference knowledge, thereby obtaining a credibility assessment result. Thus, the credibility assessment of news can be performed using the large language model and the predetermined knowledge base, thereby improving the accuracy of the news reliability assessment.

[0132] It should be noted that there are not too many restrictions on the categories of key information. For example, it can include logical reasoning content, time, place, events, people, etc.

[0133] It should be noted that, based on the semantic representation of key information, querying the reference knowledge of key information from the set knowledge base can be achieved by using any data retrieval method in the relevant technology, and no excessive restrictions are made here.

[0134] In one embodiment, based on the semantic representation of the key information, reference knowledge of the key information is queried from the setting knowledge base, including obtaining the semantic representation of the setting knowledge in the setting knowledge base, obtaining the similarity between the semantic representation of the key information and the semantic representation of the setting knowledge, and determining the setting knowledge with a similarity greater than a set threshold as reference knowledge.

[0135] It should be noted that, based on key information and reference knowledge, the credibility of the target news is evaluated and the credibility evaluation result is obtained. The relevant content of the credibility evaluation of the sample text based on the sample text and target knowledge in the above embodiment can be referred to, and will not be repeated here.

[0136] In one embodiment, the credibility assessment results include the credibility of the target news and the reasons for the credibility assessment. Therefore, the large language model in this solution can not only determine the credibility of the target news, but also the reasons for the credibility assessment, thereby increasing the diversity and comprehensiveness of the news credibility assessment results.

[0137] In one embodiment, taking the execution subject as a server as an example, after obtaining the credibility evaluation result of the target news, it also includes sending the credibility evaluation result of the target news to the client. In response, the client can receive the credibility evaluation result of the target news sent by the server, and control the display of the credibility evaluation result of the target news on the display interface of the client.

[0138] The news credibility assessment method proposed in the present disclosure obtains target news, inputs the target news into a large language model, performs credibility assessment on the target news through the large language model, and obtains a credibility assessment result of the target news. The large language model is obtained using a large language model training method, and the news credibility assessment effect of the large language model is better, which helps to improve the assessment effect of news credibility.

[0139] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0140] According to an embodiment of the present disclosure, the present disclosure also provides a large language model training device for implementing the above-mentioned large language model training method.

[0141] Figure 7 4 is a block diagram of a large language model training apparatus according to an embodiment of the present disclosure.

[0142] like Figure 7 As shown, the large language model training device 700 includes: a first acquisition module 701, a second acquisition module 702, an association module 703 and a training module 704.

[0143] The first acquisition module 701 is used to acquire a sample text containing logical reasoning content of a target reasoning method;

[0144] The second acquisition module 702 is used to obtain the credibility evaluation result of the sample text;

[0145] The association module 703 is used to associate the sample text with the credibility evaluation result of the sample text to obtain a training sample;

[0146] The training module 704 is configured to obtain a training sample set based on the plurality of training samples, and train an initial model based on the training sample set to obtain a large language model.

[0147] In one embodiment of the present disclosure, the first acquisition module 701 is further used to: obtain the target keyword for logical reasoning corresponding to the target reasoning method; determine whether the set text in the set text library contains the target keyword; if the set text contains the target keyword, determine the set text as the sample text.

[0148] In one embodiment of the present disclosure, the first acquisition module 701 is further configured to: identify candidate reasoning methods of a set text in a set text library; and if the candidate reasoning method is consistent with the target reasoning method, determine the set text as the sample text.

[0149] In one embodiment of the present disclosure, the number of target reasoning methods is N, and the training module 704 is further used to: filter out the i-th first subset from multiple training samples under the i-th target reasoning method; obtain the union of N first subsets as the training sample set; wherein N is a positive integer, and i is a positive integer not greater than N.

[0150] In one embodiment of the present disclosure, the training module 704 is also used to: determine the knowledge field to which the logical reasoning content in the sample text belongs, as the knowledge field to which the training sample corresponding to the sample text belongs; and based on the knowledge field to which the training sample belongs, filter out the training sample set from multiple training samples.

[0151] In one embodiment of the present disclosure, the training module 704 is further used to: obtain M target knowledge domains; filter out the jth second subset from multiple training samples under the jth target knowledge domain; obtain the union of the M second subsets as the training sample set; wherein M is a positive integer and j is a positive integer not greater than M.

[0152] The training device for a large language model proposed in the present disclosure obtains a sample text containing logical reasoning content of a target reasoning method, obtains a credibility assessment result of the sample text, associates the sample text with the credibility assessment result of the sample text to obtain a training sample, obtains a training sample set based on multiple training samples, and trains an initial model based on the training sample set to obtain a large language model. Thus, based on the sample text containing logical reasoning content of the target reasoning method and the credibility assessment result of the sample text, a training sample can be obtained to achieve the training of the large language model. Thus, during the training process, the large language model can learn the target reasoning method so that the credibility assessment of the text can be performed subsequently according to the target reasoning method, which helps to improve the text credibility assessment effect of the large language model and is particularly suitable for application scenarios of news credibility assessment.

[0153] According to an embodiment of the present disclosure, the present disclosure further provides a news credibility evaluation device for implementing the above-mentioned news credibility evaluation method.

[0154] Figure 8 4 is a block diagram of a device for evaluating news credibility according to an embodiment of the present disclosure.

[0155] like Figure 8 As shown, the news credibility evaluation device 800 includes: an acquisition module 801 and an evaluation module 802.

[0156] The acquisition module 801 is used to acquire target news;

[0157] The evaluation module 802 is used to input the target news into the large language model, perform credibility evaluation on the target news through the large language model, and obtain the credibility evaluation result of the target news, wherein the large language model is obtained using the large language model training method.

[0158] In one embodiment of the present disclosure, the evaluation module 802 is further used to: extract information from the target news through the large language model to obtain key information; query reference knowledge of the key information from a set knowledge base based on the semantic representation of the key information through the large language model; perform credibility evaluation on the target news based on the key information and the reference knowledge through the large language model, and obtain the credibility evaluation result.

[0159] In one embodiment of the present disclosure, the credibility evaluation result includes the credibility of the target news and the evaluation reason for the credibility.

[0160] The news credibility evaluation device proposed in the present disclosure obtains target news, inputs the target news into a large language model, performs credibility evaluation on the target news through the large language model, and obtains a credibility evaluation result of the target news. The large language model is obtained using a large language model training method, and the news credibility evaluation effect of the large language model is better, which helps to improve the evaluation effect of news credibility.

[0161] According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0162] Figure 9 A schematic block diagram of an example electronic device 900 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0163] like Figure 9 As shown, the device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 into a random access memory (RAM) 903. Various programs and data required for the operation of the device 900 can also be stored in the RAM 903. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0164] Various components in the device 900 are connected to the I / O interface 905, including an input unit 906, such as a keyboard, a mouse, etc.; an output unit 906, such as various types of displays, speakers, etc.; a storage unit 908, such as a magnetic disk, an optical disk, etc.; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the device 900 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0165] The computing unit 901 can be various general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 901 performs the various methods and processes described above, such as the large language model training method and the news credibility assessment method. For example, in some embodiments, the large language model training method and the news credibility assessment method can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as the storage unit 908. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the large language model training method and the news credibility assessment method described above can be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to execute the large language model training method and the news credibility evaluation method in any other appropriate manner (for example, by means of firmware).

[0166] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0167] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. Such program code can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0168] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0169] To initiate interaction with a user account, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user account; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user account can provide input to the computer. Other types of devices can also be used to initiate interaction with the user account; for example, feedback provided to the user account can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user account can be received in any form (including acoustic input, voice input, or tactile input).

[0170] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user account computer with a graphical user account interface or a web browser through which a user account can interact with embodiments of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0171] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0172] According to an embodiment of the present disclosure, the present disclosure also provides a computer program product, including a computer program, wherein when the computer program is executed by a processor, the steps of the large language model training method and the news credibility evaluation method described in the above embodiments of the present disclosure are implemented.

[0173] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not limited herein.

[0174] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A method for training a large language model, comprising: Obtaining sample text containing logical reasoning content of the target reasoning method; Obtaining a credibility evaluation result of the sample text; Associating the sample text with the credibility evaluation result of the sample text to obtain a training sample; Obtaining a training sample set based on the plurality of training samples, and training an initial model based on the training sample set to obtain a large language model; The number of target reasoning methods is N, and the training sample set is obtained based on the plurality of training samples, including: Select the i-th first subset from multiple training samples under the i-th target reasoning mode; Obtain the union of N first subsets as the training sample set; wherein, N is a positive integer, i is a positive integer not greater than N; or, The obtaining of a training sample set based on the plurality of training samples comprises: Determining the knowledge domain to which the logical reasoning content in the sample text belongs, as the knowledge domain to which the training sample corresponding to the sample text belongs; The training sample set is selected from a plurality of the training samples based on the knowledge domain to which the training samples belong.

2. The method according to claim 1, wherein The step of obtaining a sample text containing logical reasoning content of a target reasoning method includes: Obtaining target keywords for logical reasoning corresponding to the target reasoning method; Determining whether a setting text in a setting text library contains the target keyword; If the set text contains the target keyword, the set text is determined as the sample text.

3. The method according to claim 1, wherein The step of obtaining a sample text containing logical reasoning content of a target reasoning method includes: Identifying candidate reasoning methods for setting texts in a setting text library; If the candidate reasoning method is consistent with the target reasoning method, the set text is determined as the sample text.

4. The method according to claim 1, wherein The step of selecting the training sample set from a plurality of training samples based on the knowledge domain to which the training samples belong includes: Acquire M target knowledge domains; Select the jth second subset from multiple training samples under the jth target knowledge domain; Obtain the union of M second subsets as the training sample set; wherein, M is a positive integer, and j is a positive integer not greater than M.

5. A method for evaluating news credibility, comprising: Get targeted news; The target news is input into a large language model, the credibility of the target news is evaluated by the large language model, and a credibility evaluation result of the target news is obtained, wherein the large language model is obtained by using the large language model training method described in any one of claims 1-4.

6. The method according to claim 5, wherein: The credibility evaluation of the target news is performed using the large language model, and a credibility evaluation result of the target news is obtained, including: Extracting information from the target news using the large language model to obtain key information; querying reference knowledge of the key information from a set knowledge base based on the semantic representation of the key information through the large language model; The large language model is used to perform credibility evaluation on the target news based on the key information and the reference knowledge, and the credibility evaluation result is obtained.

7. The method according to claim 5, wherein: The credibility evaluation result includes the credibility of the target news and the evaluation reason of the credibility.

8. A large language model training device, comprising: A first acquisition module is used to acquire a sample text containing logical reasoning content of a target reasoning method; A second acquisition module is used to obtain the credibility evaluation result of the sample text; an associating module, configured to associate the sample text with the credibility evaluation result of the sample text to obtain a training sample; A training module, configured to obtain a training sample set based on the plurality of training samples, and train an initial model based on the training sample set to obtain a large language model; The number of target reasoning methods is N, and the training module is further used to: Select the i-th first subset from multiple training samples under the i-th target reasoning mode; Obtain the union of N first subsets as the training sample set; wherein, N is a positive integer, i is a positive integer not greater than N; or, The training module is further used to: Determining the knowledge domain to which the logical reasoning content in the sample text belongs, as the knowledge domain to which the training sample corresponding to the sample text belongs; The training sample set is selected from a plurality of the training samples based on the knowledge domain to which the training samples belong.

9. The device according to claim 8, wherein The first acquisition module is further configured to: Obtaining target keywords for logical reasoning corresponding to the target reasoning method; Determining whether a setting text in a setting text library contains the target keyword; If the set text contains the target keyword, the set text is determined as the sample text.

10. The device according to claim 8, wherein The first acquisition module is further configured to: Identifying candidate reasoning methods for setting texts in a setting text library; If the candidate reasoning method is consistent with the target reasoning method, the set text is determined as the sample text.

11. The device according to claim 8, wherein The training module is further used to: Acquire M target knowledge domains; Select the jth second subset from multiple training samples under the jth target knowledge domain; Obtain the union of M second subsets as the training sample set; wherein, M is a positive integer, and j is a positive integer not greater than M.

12. A device for evaluating news credibility, comprising: Acquisition module, used to obtain target news; An evaluation module is used to input the target news into a large language model, perform credibility evaluation on the target news through the large language model, and obtain a credibility evaluation result of the target news, wherein the large language model is obtained using the large language model training method described in any one of claims 1-4.

13. The device according to claim 12, wherein The evaluation module is further configured to: Extracting information from the target news using the large language model to obtain key information; querying reference knowledge of the key information from a set knowledge base based on the semantic representation of the key information through the large language model; The large language model is used to perform credibility evaluation on the target news based on the key information and the reference knowledge, and the credibility evaluation result is obtained.

14. The device according to claim 12, wherein The credibility evaluation result includes the credibility of the target news and the evaluation reason of the credibility.

15. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 7.

Images