A method and apparatus for evaluating generative model content

Abstract

The application provides an evaluation method and device for generative model content, the method comprising: analyzing the generative model content to obtain a semantic unit corresponding to the generative model content; generating a dynamic context state vector based on multi-source heterogeneous information; and obtaining a comprehensive credibility score corresponding to the generative model content according to the semantic unit corresponding to the generative model content, the dynamic context state vector, and an evaluation model, wherein the evaluation model is obtained in advance; and if it is determined that the comprehensive credibility score corresponding to the generative model content is less than a threshold value, intervening in the output of the generative model content based on the comprehensive credibility score corresponding to the generative model content. The device is used to execute the above method. The evaluation method and device for generative model content provided in the embodiments of the application improve the reliability of the evaluation of generative model content.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and specifically to a method and apparatus for evaluating the content of generative models. Background Technology

[0002] Generative artificial intelligence is widely used in fields such as text generation, dialogue interaction, content creation, and code synthesis.

[0003] In existing technologies, generative models sometimes produce outdated, disconnected, or context-inappropriate output. Therefore, it is necessary to evaluate the output of generative models. Currently, the evaluation of generative content primarily focuses on hallucination detection. Developing a more reliable method for evaluating the content of generative models has become a crucial issue that urgently needs to be addressed in this field. Summary of the Invention

[0004] To address the problems in the prior art, embodiments of the present invention provide a method and apparatus for evaluating generative model content, which can at least partially solve the problems existing in the prior art.

[0005] In a first aspect, the present invention proposes a method for evaluating the content of a generative model, comprising:

[0006] The content of the generative model is parsed to obtain the semantic units corresponding to the content of the generative model;

[0007] Based on multi-source heterogeneous information, a dynamic context state vector is generated.

[0008] Based on the semantic units corresponding to the content of the generative model, the dynamic context state vector, and the evaluation model, a comprehensive credibility score corresponding to the content of the generative model is obtained; wherein, the evaluation model is obtained in advance;

[0009] If it is determined that the overall credibility score corresponding to the generative model content is less than a threshold, then the output of the generative model content is intervened based on the overall credibility score corresponding to the generative model content.

[0010] Secondly, the present invention provides an evaluation apparatus for the content of a generative model, comprising:

[0011] The parsing module is used to parse the content of the generative model to obtain the semantic units corresponding to the content of the generative model.

[0012] The generation module is used to generate dynamic contextual state vectors based on multi-source heterogeneous information;

[0013] The scoring module is used to obtain a comprehensive credibility score for the content of the generative model based on the semantic units corresponding to the content of the generative model, the dynamic context state vector, and the evaluation model; wherein the evaluation model is obtained in advance.

[0014] The intervention module is used to intervene in the output of the generative model content based on the comprehensive credibility score corresponding to the generative model content after determining that the comprehensive credibility score is less than a threshold.

[0015] Thirdly, the present invention provides a computer device including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the program to implement the evaluation method for generative model content as described in any of the above embodiments.

[0016] Fourthly, the present invention provides a computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the evaluation method for generative model content described in any of the above embodiments.

[0017] Fifthly, the present invention provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the evaluation method for generative model content described in any of the above embodiments.

[0018] The method and apparatus for evaluating generative model content provided in this invention parses the generative model content to obtain semantic units corresponding to the generative model content; generates dynamic contextual state vectors based on multi-source heterogeneous information; and obtains a comprehensive credibility score corresponding to the generative model content based on the semantic units corresponding to the generative model content, the dynamic contextual state vectors, and the evaluation model; wherein the evaluation model is obtained in advance; if it is determined that the comprehensive credibility score corresponding to the generative model content is less than a threshold, the output of the generative model content is intervened based on the comprehensive credibility score corresponding to the generative model content. By introducing dynamic contextual state vectors to evaluate the generative model content, the reliability of the generative model content evaluation is improved. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0020] Figure 1This is a flowchart illustrating the evaluation method for generative model content provided in the first embodiment of the present invention.

[0021] Figure 2 This is a flowchart illustrating the evaluation method for generative model content provided in the second embodiment of the present invention.

[0022] Figure 3 This is a flowchart illustrating the evaluation method for generative model content provided in the third embodiment of the present invention.

[0023] Figure 4 This is a schematic diagram of the structure of the generative model content evaluation device provided in the fourth embodiment of the present invention.

[0024] Figure 5 This is a schematic diagram of the structure of the generative model content evaluation device provided in the fifth embodiment of the present invention.

[0025] Figure 6 This is a schematic diagram of the structure of the generative model content evaluation device provided in the sixth embodiment of the present invention.

[0026] Figure 7 This is a schematic diagram of the structure of the generative model content evaluation device provided in the seventh embodiment of the present invention.

[0027] Figure 8 This is a schematic diagram of the physical structure of the computer device provided in the eighth embodiment of the present invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. Here, the illustrative embodiments and their descriptions are used to explain the present invention, but are not intended to limit the present invention. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be arbitrarily combined with each other. The acquisition, storage, use, and processing of data in the technical solutions of this application all comply with relevant laws and regulations. The user information in the embodiments of this application is obtained through legal and compliant means, and the acquisition, storage, use, and processing of user information have been authorized and agreed upon by the customer.

[0029] To facilitate understanding of the technical solution provided in this application, the relevant content of the technical solution in this application will be explained below.

[0030] Generative models: These are artificial intelligence models that can generate text, images, audio, code, and other content based on input (such as prompts or context). They include, but are not limited to, large language models, multimodal generative models, and code-generating models.

[0031] Semantic unit: refers to the smallest evaluable unit of information obtained by structured parsing of the output of a generative model, which contains elements that represent the core semantics of the output.

[0032] In practical applications, the output of generative models (hereinafter referred to as generative model content) is affected by dynamically changing contexts, including but not limited to: real-time shifts in user intent, continuous updates from external data streams, and changes in domain knowledge and regulatory standards. These changes can lead to outdated, disconnected, or context-incompatible outputs from the model, thereby raising decision-making risks and security issues.

[0033] Existing technologies have limitations in evaluating the content of generative models: lack of context awareness: unable to model and respond to dynamically evolving context states in real time; lack of predictive ability: not covering the ability to predict the credibility decay of output under future context changes; lack of closed-loop mechanism: lacking a feedback learning loop for self-learning optimization based on historical decision results.

[0034] Therefore, this application proposes an evaluation method for generative model content, aiming to overcome the fundamental defects of existing generative model content evaluation technologies, such as staticity, passivity, and non-evolution, and to improve the reliability, timeliness, and compliance of generated content in complex and ever-changing environments.

[0035] Figure 1 This is a flowchart illustrating an evaluation method for generative model content provided in an embodiment of the present invention, as shown below. Figure 1 As shown, the method for evaluating the content of a generative model provided in this embodiment of the invention includes:

[0036] S101. Parse the content of the generative model to obtain the semantic units corresponding to the content of the generative model;

[0037] Specifically, generative model content refers to text or unstructured content output by a generative model. Parsing the generative model content yields semantic units corresponding to it. These semantic units are structured data that can be processed by a computer and reflect the core semantic elements of the generative model content. The execution entity of the generative model content evaluation method provided in this embodiment includes, but is not limited to, a server.

[0038] For example, a semantic unit includes a semantic triple [subject-predicate-object]. The subject refers to the entity that performs the action in the content of the generative model, the object refers to the object on which the action is performed in the content of the generative model, and the predicate is the action in the content of the generative model.

[0039] S102. Generate dynamic context state vectors based on multi-source heterogeneous information;

[0040] Specifically, based on multi-source heterogeneous information, a dynamic context state vector is generated. This dynamic context state vector is a numerical representation that comprehensively reflects changes in the context and is used to provide a dynamic benchmark for evaluating the content of the generative model. The multi-source heterogeneous information includes, but is not limited to, user interaction information, external data streams, and domain information knowledge bases.

[0041] User interaction information includes, but is not limited to, the user's historical dialogue with the generative model before the model outputs content, input intent information, and user behavior sequences. A user behavior sequence refers to a record of interactive behaviors generated during user interactions with the generative model within a preset time window, arranged chronologically, used to characterize the evolution of the user's intent and changes in context during multiple rounds of interaction. The user behavior sequence includes, but is not limited to, historical input content submitted by the user to the generative model, corresponding time information, follow-up questions regarding the generative model's output, modification of prompts, regeneration requests, confirmation or negative feedback, and other interactive behaviors.

[0042] External data streams include, but are not limited to, real-time financial market data, meteorological data, policy and regulatory information, etc., and can be relevant information obtained by generative models from the Internet based on user interaction information.

[0043] The domain knowledge base information includes, but is not limited to, standard operating procedures, legal provisions, technical terms, and parameter libraries, and can be set according to actual needs; this embodiment of the invention does not impose any limitations. Relevant domain knowledge base information can be queried based on user interaction information.

[0044] S103. Based on the semantic units corresponding to the content of the generative model, the dynamic context state vector, and the evaluation model, obtain the comprehensive credibility score corresponding to the content of the generative model; wherein, the evaluation model is obtained in advance.

[0045] Specifically, based on the semantic units corresponding to the generative model content, the dynamic context state vector, and the evaluation model, a comprehensive credibility score corresponding to the generative model content can be obtained. This comprehensive credibility score is used to evaluate the credibility of the generative model content to identify potential risks. The evaluation model is obtained in advance.

[0046] For example, historical generative model content and corresponding dynamic context state vectors can be collected as training data, and the comprehensive credibility score corresponding to the historical generative model content can be used as a label to train the original model and obtain the evaluation model. The original model can be a deep neural network model or a multi-task neural network model, selected according to actual needs; this embodiment of the invention does not impose any limitations.

[0047] For example, by inputting the semantic units corresponding to the generative model content and the dynamic context state vector into the evaluation model, a comprehensive credibility score corresponding to the generative model content can be output.

[0048] S104. If it is determined that the overall credibility score corresponding to the generative model content is less than the intervention threshold, then the output of the generative model content is intervened based on the overall credibility score corresponding to the generative model content.

[0049] Specifically, the overall credibility score corresponding to the generative model content is compared with an intervention threshold. If the overall credibility score is greater than or equal to the intervention threshold, the generative model content can be directly output for user reference. If the overall credibility score is less than the intervention threshold, it indicates that the credibility of the generative model content is questionable, and intervention is needed on the output of the generative model content. Intervention can be based on the overall credibility score corresponding to the generative model content. The intervention threshold is set based on practical experience, and this embodiment of the invention does not impose any limitations.

[0050] The method for evaluating generative model content provided in this invention involves parsing the generative model content to obtain semantic units corresponding to the content; generating dynamic contextual state vectors based on multi-source heterogeneous information; obtaining a comprehensive credibility score for the generative model content based on the semantic units, the dynamic contextual state vectors, and the evaluation model; wherein the evaluation model is pre-obtained; if it is determined that the comprehensive credibility score for the generative model content is less than a threshold, the output of the generative model content is intervened based on the comprehensive credibility score. By introducing dynamic contextual state vectors to evaluate the generative model content, the reliability of the evaluation is improved.

[0051] Based on the above embodiments, further, the step of parsing the generative model content to obtain the semantic units corresponding to the generative model content includes:

[0052] The generative model content is input into the relation extraction model, and the semantic units corresponding to the generative model content are output; wherein, the relation extraction model is obtained in advance.

[0053] Specifically, inputting the generative model content into the relation extraction model enables the output of semantic units corresponding to the generative model content. The relation extraction model is obtained in advance.

[0054] For example, text corpora, including but not limited to industry documents, dialogue logs, and operating procedures, can be collected as training data. A self-supervised learning model can be used to generate labels (i.e., semantic units) corresponding to the training data. The deep neural network model can then be trained based on the training data and the corresponding labels to obtain a general language understanding model. The self-supervised learning model can be a Masked Language Modeling (MLM) model or a NextSentence Prediction (NSP) model. The deep neural network model can be a Bidirectional Encoder Representations from Transformers (BERT) model, a Robustly Optimized BERT Pretraining (RoBERTa) model, a Generative Pre-trained Transformer (GPT) model, etc., selected according to the actual situation. This embodiment of the invention does not impose any limitations.

[0055] After obtaining a general language understanding model, it can be fine-tuned to obtain a relation extraction model, thereby improving the accuracy of the relation extraction model. Fine-tuning involves continuing training on the language understanding model using task data, updating all or some of its parameters to make it more suitable for parsing generative model content. The task data is set according to actual needs, and this embodiment of the invention does not impose any limitations.

[0056] Figure 2 This is a flowchart illustrating the evaluation method for generative model content provided in the third embodiment of the present invention, as shown below. Figure 2 As shown, based on the above embodiments, the step of generating a dynamic contextual state vector based on multi-source heterogeneous information further includes:

[0057] S201. Perform feature encoding on the multi-source heterogeneous information to obtain the vector representation corresponding to the multi-source heterogeneous information;

[0058] Specifically, feature encoding of the multi-source heterogeneous information can yield a vector representation of the multi-source heterogeneous information. Different information can employ different encoding methods, selected according to actual needs; this embodiment of the invention does not impose limitations. The specific process of feature encoding is prior art and will not be elaborated here. After feature encoding, different information can be represented by vectors of a unified dimension, thus obtaining the vector representation of the multi-source heterogeneous information.

[0059] For textual information, a pre-trained language model can be used for encoding; for numerical time series, it can be standardized and then encoded using a fully connected or RNN; for graph-structured knowledge bases, a graph neural network (GNN) can be used for encoding.

[0060] S202. The vector representations corresponding to the multi-source heterogeneous information are spliced ​​or fused to obtain the original dynamic context state vectors corresponding to the multi-source heterogeneous information.

[0061] Specifically, the vector representations corresponding to the multi-source heterogeneous information are feature-fused to form a unified vector representation, which serves as the original dynamic context state vector corresponding to the multi-source heterogeneous information.

[0062] Feature fusion can be achieved through vector concatenation or attention-based fusion. Vector concatenation involves joining multiple vector representations along a unified dimension. Attention-based fusion can weight multiple vector representations and use the weighted result as the original dynamic context state vector.

[0063] S203. Obtain the dynamic context state vector based on the original dynamic context state vector corresponding to the multi-source heterogeneous information and the temporal coding model.

[0064] Specifically, the original dynamic context state vector corresponding to the multi-source heterogeneous information is input into a temporal coding model, and the dynamic context state vector is output. The temporal coding model is pre-trained.

[0065] For example, the original dynamic context state vectors corresponding to historical multi-source heterogeneous information are collected as training data, and the dynamic context state vectors corresponding to the original dynamic context state vectors corresponding to historical multi-source heterogeneous information are used as labels to train the initial temporal coding model, thus obtaining the temporal coding model. The temporal coding model can employ a Long Short-Term Memory (LSTM) network model, a Gated Recurrent Unit (GRU) model, a Transformer model, etc., selected according to actual needs; this embodiment of the invention does not impose any limitations.

[0066] Figure 3 This is a flowchart illustrating the evaluation method for generative model content provided in the third embodiment of the present invention, as shown below. Figure 3 As shown, based on the above embodiments, further, obtaining the comprehensive credibility score corresponding to the generative model content based on the semantic units corresponding to the generative model content, the dynamic context state vector, and the evaluation model includes:

[0067] S301. Input the semantic units corresponding to the content of the generative model and the dynamic context state vector into the evaluation model, and output multiple evaluation indicators.

[0068] Specifically, by inputting the semantic units corresponding to the content of the generative model and the dynamic context state vector into the evaluation model, multiple evaluation metrics can be output. These evaluation metrics can be set according to actual needs, and this embodiment of the invention does not impose any limitations.

[0069] For example, several evaluation metrics include contextual consistency, contextual sensitivity risk score, and adaptability bias. Contextual consistency characterizes the degree to which the content of the generative model matches the current context; contextual sensitivity risk score characterizes the probability that the credibility of the content of the generative model will decrease under future contextual changes; and adaptability bias measures the degree of deviation of the content of the generative model from current and newly emerging contextual elements, reflecting whether the evaluation model relies excessively on historical contextual patterns or inherent assumptions.

[0070] S302. Based on each evaluation indicator and its corresponding weight, obtain the comprehensive credibility score corresponding to the content of the generative model.

[0071] Specifically, the sum of the products of each evaluation index and its corresponding weight is calculated as the comprehensive credibility score corresponding to the content of the generative model.

[0072] Based on the above embodiments, the further step of intervening in the output of the generative model content based on the comprehensive credibility score corresponding to the generative model content includes:

[0073] If it is determined that the overall credibility score corresponding to the generative model content is less than the first preset value, then the output of the generative model content is prevented.

[0074] If it is determined that the overall credibility score corresponding to the generative model content is greater than or equal to the first preset value and less than the second preset value, then the generative model content is sent for review.

[0075] If it is determined that the overall credibility score corresponding to the generative model content is greater than or equal to the second preset value and less than the intervention threshold, then the generative model content is output along with a prompt message.

[0076] Specifically, the overall credibility score corresponding to the generative model content is compared with a first preset value. If the overall credibility score is less than the first preset value, it indicates that the credibility of the generative model content is very low or obviously violates the rules. In this case, the output of the generative model content is blocked, that is, the generative model content will not be provided to the user.

[0077] If the overall credibility score is greater than or equal to the first preset value and less than the second preset value, it indicates that the generative model content needs further evaluation before being provided to the user. In this case, the generative model content is sent to a human review or expert system for verification. If the verification passes, the generative model content is output; if the verification fails, the output of the generative model content is prevented.

[0078] If the overall credibility score is greater than or equal to the second preset value but less than the intervention threshold, it indicates that the generative model content poses a risk. In this case, the generative model content can be output along with a prompt message. The prompt message can be set according to actual needs, and this embodiment of the invention does not impose any limitations.

[0079] For example, the warning information may include additional risk warnings and / or disclaimers.

[0080] It can record intervention-related information for generative model content, including but not limited to the original comprehensive credibility score and various evaluation indicators; the triggered intervention level and specific actions; and the actual results after the intervention, such as whether the risk was avoided or whether misjudgments occurred. Recording the above intervention information in the database can be used to optimize the evaluation model.

[0081] Based on the above embodiments, the evaluation method for generative model content provided in this embodiment of the invention further includes:

[0082] Based on the comprehensive credibility score corresponding to the generative model content and the intervention feedback results of the generative model content, adjust at least one of the first preset value, the second preset value, and the intervention threshold.

[0083] Specifically, at least one of the first preset value, the second preset value, and the intervention threshold can be adjusted based on the comprehensive credibility score corresponding to the generative model content and the intervention feedback result of the generative model content. The intervention feedback result of the generative model content can be an effective intervention or an ineffective intervention.

[0084] The first preset value, the second preset value, and the intervention threshold can change dynamically over time; different first preset values, second preset values, and intervention thresholds can be set for different business scenarios; different first preset values, second preset values, and intervention thresholds can be set according to user type or content category. The update cycle of the first preset value, the second preset value, and the intervention threshold can be configured.

[0085] For example, threshold updates can be achieved using a sliding window statistical method. This involves collecting the overall credibility score of each generative model's content and its corresponding intervention feedback results within a set sliding time window; obtaining the distribution of the overall credibility scores for each generative model's content within the set sliding time window; and, based on the intervention feedback results corresponding to each threshold interval, calculating the effective intervention percentage for each threshold interval. The threshold intervals are defined as: less than a first preset value, greater than or equal to the first preset value and less than a second preset value, and greater than or equal to the second preset value and less than the intervention threshold.

[0086] Determine whether the effective intervention percentage corresponding to each threshold interval meets the corresponding requirements. For example, if the effective intervention percentage corresponding to each threshold interval is greater than 95%, and the requirement is not met, adjust the threshold intervals so that the effective intervention percentage of the corresponding threshold interval meets the corresponding requirements. If not all threshold intervals can meet the corresponding requirements, prioritize ensuring that the main threshold intervals meet the requirements. For example, threshold intervals less than the first preset value have higher priority than threshold intervals greater than or equal to the second preset value and less than the intervention threshold. Threshold intervals greater than or equal to the second preset value and less than the intervention threshold have higher priority than threshold intervals greater than or equal to the first preset value and less than the second preset value.

[0087] For example, the first preset value, the second preset value, and the intervention threshold can be updated using a Bayesian update method. By using historical first preset values, historical second preset values, and historical intervention thresholds as priors, and then updating these values ​​posteriorly based on newly observed effective interventions, stability can be improved.

[0088] For example, reinforcement learning strategies can be used to update the first preset value, the second preset value, and the intervention threshold. Using the effective intervention rate and the false intervention rate as rewards, the optimal threshold strategy is learned to maximize long-term risk control benefits.

[0089] Figure 4 This is a schematic diagram of the structure of the evaluation device for generative model content provided in the fourth embodiment of the present invention, as shown below. Figure 4 As shown, the generative model content evaluation device provided in this embodiment of the invention includes a parsing module 401, a generation module 402, a scoring module 403, and an intervention module 404, wherein:

[0090] The parsing module 401 is used to parse the content of the generative model to obtain the semantic units corresponding to the content of the generative model; the generation module 402 is used to generate a dynamic context state vector based on multi-source heterogeneous information; the scoring module 403 is used to obtain a comprehensive credibility score corresponding to the content of the generative model based on the semantic units corresponding to the content of the generative model, the dynamic context state vector, and the evaluation model; wherein, the evaluation model is obtained in advance; the intervention module 404 is used to intervene in the output of the content of the generative model based on the comprehensive credibility score corresponding to the content of the generative model after determining that the comprehensive credibility score corresponding to the content of the generative model is less than a threshold.

[0091] Specifically, generative model content refers to text or unstructured content output by a generative model. The parsing module 401 parses the generative model content to obtain the semantic units corresponding to it. These semantic units are structured data that can be processed by a computer and reflect the core semantic elements of the generative model content.

[0092] The generation module 402 generates a dynamic context state vector based on multi-source heterogeneous information. This dynamic context state vector is a numerical representation that comprehensively reflects changes in the context, providing a dynamic benchmark for evaluating the content of the generative model. The multi-source heterogeneous information includes, but is not limited to, user interaction information, external data streams, and domain information knowledge bases.

[0093] The scoring module 403, based on the semantic units corresponding to the generative model content, the dynamic context state vector, and the evaluation model, can obtain a comprehensive credibility score for the generative model content. This comprehensive credibility score is used to assess the credibility of the generative model content to identify potential risks. The evaluation model is obtained in advance.

[0094] The intervention module 404 compares the overall credibility score corresponding to the generative model content with an intervention threshold. If the overall credibility score is greater than or equal to the intervention threshold, the generative model content can be directly output for user reference. If the overall credibility score is less than the intervention threshold, it indicates that the credibility of the generative model content is questionable, and intervention is needed on the output of the generative model content. Intervention can be based on the overall credibility score corresponding to the generative model content. The intervention threshold is set based on practical experience, and this embodiment of the invention does not impose any limitations.

[0095] The generative model content evaluation device provided in this embodiment of the invention parses the generative model content to obtain the semantic units corresponding to the generative model content; generates a dynamic context state vector based on multi-source heterogeneous information; and obtains a comprehensive credibility score corresponding to the generative model content based on the semantic units corresponding to the generative model content, the dynamic context state vector, and the evaluation model; wherein, the evaluation model is obtained in advance; if it is determined that the comprehensive credibility score corresponding to the generative model content is less than a threshold, the output of the generative model content is intervened based on the comprehensive credibility score corresponding to the generative model content. By introducing the dynamic context state vector to evaluate the generative model content, the reliability of the generative model content evaluation is improved.

[0096] Based on the above embodiments, the parsing module 401 is further specifically used for:

[0097] The generative model content is input into the relation extraction model, and the semantic units corresponding to the generative model content are output; wherein, the relation extraction model is obtained in advance.

[0098] Figure 5 This is a schematic diagram of the structure of the evaluation device for generative model content provided in the fifth embodiment of the present invention, as shown below. Figure 5 As shown, based on the above embodiments, the generation module 402 further includes a feature encoding unit 4021, a feature fusion unit 4022, and a first obtaining unit 4023, wherein:

[0099] The feature encoding unit 4021 is used to encode the multi-source heterogeneous information to obtain the vector representation corresponding to the multi-source heterogeneous information; the feature fusion unit 4022 is used to fuse the vector representation corresponding to the multi-source heterogeneous information to obtain the original dynamic context state vector corresponding to the multi-source heterogeneous information; the first obtaining unit 4023 is used to obtain the dynamic context state vector according to the original dynamic context state vector corresponding to the multi-source heterogeneous information and the temporal coding model.

[0100] Figure 6 This is a schematic diagram of the structure of the evaluation device for generative model content provided in the sixth embodiment of the present invention, as shown below. Figure 6 As shown, based on the above embodiments, the scoring module 403 further includes an output unit 4031 and a second obtaining unit 4032, wherein:

[0101] The output unit 4031 is used to input the semantic unit corresponding to the content of the generative model and the dynamic context state vector into the evaluation model and output multiple evaluation indicators; the second obtaining unit 4032 is used to obtain the comprehensive credibility score corresponding to the content of the generative model according to each evaluation indicator and its corresponding weight.

[0102] Based on the above embodiments, the intervention module 404 is further specifically used for:

[0103] If it is determined that the overall credibility score corresponding to the generative model content is less than the first preset value, then the output of the generative model content is prevented.

[0104] If it is determined that the overall credibility score corresponding to the generative model content is greater than or equal to the first preset value and less than the second preset value, then the generative model content is sent for review.

[0105] If it is determined that the overall credibility score corresponding to the generative model content is greater than or equal to the second preset value and less than the intervention threshold, then the generative model content is output along with a prompt message.

[0106] Figure 7 This is a schematic diagram of the structure of the generative model content evaluation device provided in the seventh embodiment of the present invention, as shown below. Figure 7 As shown, based on the above embodiments, the generative model content evaluation device provided in this embodiment of the invention further includes an adjustment module 405, wherein:

[0107] The adjustment module 405 is used to adjust the first preset value, the second preset value, and the intervention threshold based on the comprehensive credibility score corresponding to the generative model content and the intervention feedback result of the generative model content.

[0108] The embodiments of the device provided in this invention can be used to execute the processing flow of the above-described method embodiments. Its functions will not be repeated here, but can be referred to the detailed description of the above-described method embodiments.

[0109] Figure 8 This is a schematic diagram of the physical structure of the computer device provided in the eighth embodiment of the present invention, as shown below. Figure 8As shown, the computer device may include a processor 801, a communications interface 802, a memory 803, and a communication bus 804, wherein the processor 801, communications interface 802, and memory 803 communicate with each other via the communication bus 804. The processor 801 can call logical instructions in the memory 803 to execute the methods provided in the above-described method embodiments, such as: parsing the generative model content to obtain semantic units corresponding to the generative model content; generating a dynamic context state vector based on multi-source heterogeneous information; obtaining a comprehensive credibility score corresponding to the generative model content based on the semantic units corresponding to the generative model content, the dynamic context state vector, and an evaluation model; wherein the evaluation model is pre-obtained; if it is determined that the comprehensive credibility score corresponding to the generative model content is less than a threshold, then the output of the generative model content is intervened based on the comprehensive credibility score corresponding to the generative model content.

[0110] Furthermore, the logical instructions in the aforementioned memory 803 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0111] This embodiment discloses a computer program product, which includes a computer program / instructions stored on a computer-readable storage medium. When the computer program / instructions are executed by a computer, the computer can perform the methods provided in the above-described method embodiments, including, for example: parsing the content of a generative model to obtain semantic units corresponding to the content of the generative model; generating a dynamic context state vector based on multi-source heterogeneous information; obtaining a comprehensive credibility score corresponding to the content of the generative model based on the semantic units corresponding to the content of the generative model, the dynamic context state vector, and an evaluation model; wherein the evaluation model is obtained in advance; if it is determined that the comprehensive credibility score corresponding to the content of the generative model is less than a threshold, then the output of the content of the generative model is intervened based on the comprehensive credibility score corresponding to the content of the generative model.

[0112] This embodiment provides a computer-readable storage medium storing a computer program / instruction. When the computer program / instruction is executed by a processor, it causes the computer to perform the methods provided in the above-described method embodiments, including, for example: parsing the content of a generative model to obtain semantic units corresponding to the content of the generative model; generating a dynamic context state vector based on multi-source heterogeneous information; obtaining a comprehensive credibility score corresponding to the content of the generative model based on the semantic units corresponding to the content of the generative model, the dynamic context state vector, and an evaluation model; wherein the evaluation model is obtained in advance; if it is determined that the comprehensive credibility score corresponding to the content of the generative model is less than a threshold, then the output of the content of the generative model is intervened based on the comprehensive credibility score corresponding to the content of the generative model.

[0113] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0114] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0115] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0116] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0117] In the description of this specification, the references to terms such as "an embodiment," "a specific embodiment," "some embodiments," "for example," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0118] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for evaluating the content of a generative model, characterized in that, include: The content of the generative model is parsed to obtain the semantic units corresponding to the content of the generative model; Based on multi-source heterogeneous information, a dynamic context state vector is generated. Based on the semantic units corresponding to the content of the generative model, the dynamic context state vector, and the evaluation model, a comprehensive credibility score corresponding to the content of the generative model is obtained; wherein, the evaluation model is obtained in advance; If it is determined that the overall credibility score corresponding to the generative model content is less than a threshold, then the output of the generative model content is intervened based on the overall credibility score corresponding to the generative model content.

2. The method according to claim 1, characterized in that, The step of parsing the generative model content to obtain the semantic units corresponding to the generative model content includes: The generative model content is input into the relation extraction model, and the semantic units corresponding to the generative model content are output; wherein, the relation extraction model is obtained in advance.

3. The method according to claim 1, characterized in that, The generation of dynamic context state vectors based on multi-source heterogeneous information includes: The multi-source heterogeneous information is feature-encoded to obtain a vector representation corresponding to the multi-source heterogeneous information; Feature fusion is performed on the vector representations corresponding to the multi-source heterogeneous information to obtain the original dynamic context state vectors corresponding to the multi-source heterogeneous information; The dynamic context state vector is obtained based on the original dynamic context state vector corresponding to the multi-source heterogeneous information and the temporal coding model.

4. The method according to claim 1, characterized in that, The step of obtaining the comprehensive credibility score corresponding to the content of the generative model based on the semantic units corresponding to the content of the generative model, the dynamic context state vector, and the evaluation model includes: The semantic units corresponding to the content of the generative model and the dynamic context state vector are input into the evaluation model, and multiple evaluation indicators are output. Based on each evaluation indicator and its corresponding weight, a comprehensive credibility score is obtained for the content of the generative model.

5. The method according to any one of claims 1 to 4, characterized in that, The intervention on the output of the generative model content based on the comprehensive credibility score corresponding to the generative model content includes: If it is determined that the overall credibility score corresponding to the generative model content is less than the first preset value, then the output of the generative model content is prevented. If it is determined that the overall credibility score corresponding to the generative model content is greater than or equal to the first preset value and less than the second preset value, then the generative model content is sent for review. If it is determined that the overall credibility score corresponding to the generative model content is greater than or equal to the second preset value and less than the intervention threshold, then the generative model content is output along with a prompt message.

6. The method according to claim 5, characterized in that, Also includes: The first preset value, the second preset value, and the intervention threshold are adjusted based on the comprehensive credibility score corresponding to the generative model content and the intervention feedback results of the generative model content.

7. An evaluation device for generative model content, characterized in that, include: The parsing module is used to parse the content of the generative model to obtain the semantic units corresponding to the content of the generative model. The generation module is used to generate dynamic contextual state vectors based on multi-source heterogeneous information; The scoring module is used to obtain a comprehensive credibility score for the content of the generative model based on the semantic units corresponding to the content of the generative model, the dynamic context state vector, and the evaluation model; wherein the evaluation model is obtained in advance. The intervention module is used to intervene in the output of the generative model content based on the comprehensive credibility score corresponding to the generative model content after determining that the comprehensive credibility score is less than a threshold.

8. A computer device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program / instructions that, when executed by a processor, implement the method according to any one of claims 1 to 6.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the method described in any one of claims 1 to 6.

Images