Method and device for inhibiting hallucination of large model and storage medium
By masking the original data of the large model and generating negative question-answer pairs, the initial large model was adjusted, which solved the problem of the large model producing illusions due to data defects. It was realized that when defective data is received, negative answers are output to avoid illusions.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ZHEJIANG DAHUA TECH CO LTD
- Filing Date
- 2025-12-16
- Publication Date
- 2026-05-12
AI Technical Summary
Large models may produce fictitious, erroneous, or contradictory information in their output due to model mechanisms or data defects, and existing methods cannot solve the illusion problem at its source.
By masking the obtained raw data to be queried, negative question-answer pairs are generated, and the initial large model is adjusted to generate the target large model, thus avoiding fictitious data.
The target large model can output a negative answer when it receives a query question with defective data, thus avoiding the illusion problem of generating a large model.
Smart Images
Figure CN122021871A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, and storage medium for suppressing large model illusions. Background Technology
[0002] Currently, large-scale modeling technology is developing rapidly in society, bringing efficiency improvements to people's learning and work with its powerful text, image, and video generation capabilities. More and more people are using large-scale models in their daily lives. However, large models may produce fictitious, erroneous, or contradictory information in their output due to model mechanisms or data defects, which is known as the "large model illusion." For example, in the NL2SQL task, large language models (LLMs) often lack modeling of "unknown fields / unknown tables" in the training data, causing them to fabricate fields and / or tables that do not exist in the database and output them during inference.
[0003] In general, large model inferences use data augmentation retrieval or post-inference verification to mitigate hallucinations, but these methods do not fundamentally solve the hallucination problem at its source. Summary of the Invention
[0004] This application provides at least one method, apparatus, device, and computer-readable storage medium for suppressing large model illusions.
[0005] The first aspect of this application provides a method for suppressing the illusion of a large model, comprising: masking the acquired raw data to be queried to obtain masked data; generating a negative question-answer pair based on the masked data, the negative question-answer pair being used to characterize the defects in the masked data; and adjusting the acquired initial large model based on the negative question-answer pair and the masked data to obtain a target large model.
[0006] In one embodiment, the step of masking the acquired raw data to be queried to obtain masked data includes: acquiring pre-labeling information of the raw data, the pre-labeling information including required data and / or optional data; and masking the optional data in the raw data to obtain the masked data.
[0007] In one embodiment, the step of masking optional data in the original data to obtain the masked data includes: obtaining a preset masking ratio; and masking target data in the optional data according to the preset masking ratio to obtain the masked data.
[0008] In one embodiment, the step of masking the target data in the optional data according to the preset masking ratio to obtain the masked data includes: obtaining the associated data of the target data; and masking the target data and the associated data to obtain the masked data.
[0009] In one embodiment, the step of masking target data in the optional data according to the preset masking ratio to obtain the masked data includes: obtaining the data importance of the optional data; determining the target data in the optional data according to the data importance; and masking the target data to obtain the masked data.
[0010] In one embodiment, generating a negative question-answer pair based on the masked data includes: generating a natural language question and a negative answer corresponding to the natural language question based on the masked data, wherein the negative answer indicates that the masked data has a defect when used to solve the natural language question; and determining the natural language question and the negative answer as the negative question-answer pair.
[0011] In one embodiment, adjusting the initial large model obtained based on the negative question-answer pairs and the masked data to obtain the target large model includes: organizing multiple masked data and corresponding negative question-answer pairs into a negative sample set; and performing model fine-tuning processing on the initial large model based on the negative sample set to obtain the target large model.
[0012] In one embodiment, adjusting the initial large model obtained based on the negative question-answer pairs and the masked data to obtain the target large model includes: organizing multiple masked data and corresponding negative question-answer pairs into a negative sample set; obtaining the positive sample set corresponding to the original data; and performing model fine-tuning processing on the initial large model based on the negative sample set and the positive sample set to obtain the target large model.
[0013] A second aspect of this application provides an apparatus for suppressing the illusion of a large model, comprising: a data masking module for masking acquired raw data to be queried to obtain masked data; a question-and-answer generation module for generating negative question-and-answer pairs based on the masked data, the negative question-and-answer pairs being used to characterize the defects in the masked data; and a model adjustment module for adjusting the acquired initial large model based on the negative question-and-answer pairs and the masked data to obtain a target large model.
[0014] A third aspect of this application provides an electronic device including a memory and a processor, the processor being configured to execute program instructions stored in the memory to implement the aforementioned method for suppressing large model illusions.
[0015] The fourth aspect of this application provides a computer-readable storage medium having program instructions stored thereon, which, when executed by a processor, implement the above-described method for suppressing large model illusions.
[0016] The above scheme obtains raw data, such as from a database, and masks the table and / or field information of the raw data to obtain masked data with data defects. Based on the masked data, corresponding negative question-and-answer pairs are generated. These pairs include a natural language question that might be posed to the masked data and a negative answer indicating that the masked data is defective and used to solve that question. The initial large model (such as an NL2SQL generative model) obtained from the negative question-and-answer pairs and the masked data is trained and adjusted to obtain the target large model. This allows the target large model to output a negative answer (e.g., indicating that certain data in the query is unavailable or does not exist) when receiving a query containing defective data in practical applications, instead of generating a positive answer to solve the query as is the case with traditional generative large models. This avoids the illusion problem that generative large models may produce during data generation.
[0017] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. Attached Figure Description
[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.
[0019] Figure 1 This is a flowchart illustrating an exemplary embodiment of the method for suppressing large model illusions according to this application; Figure 2 This is an exemplary overall process diagram of a method for suppressing large model illusions according to this application; Figure 3 This is an exemplary system architecture diagram of a method for suppressing large model illusion in this application; Figure 4 This is a block diagram illustrating a device for suppressing large model illusions, as shown in an exemplary embodiment of this application; Figure 5 This is a schematic diagram of the structure of an embodiment of the electronic device of this application; Figure 6 This is a schematic diagram of the structure of an embodiment of the computer-readable storage medium of this application. Detailed Implementation
[0020] The embodiments of this application will now be described in detail with reference to the accompanying drawings.
[0021] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.
[0022] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this document means two or more. Moreover, the term "at least one" in this document means any combination of at least two of any one or more of a plurality of objects. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.
[0023] To facilitate understanding, one of the applicable scenarios of this application will be illustrated by example.
[0024] The "big model illusion" (also known as AI illusion) typically refers to the phenomenon where large language models fabricate information they believe to be real, or even seemingly reasonable or credible, ultimately leading to discrepancies between the content generated by the big model and real-world facts or user input. The main reasons for this illusion include data defects, low utilization of factual knowledge captured in the data, insufficient recall of long-tail knowledge, and difficulty in handling complex reasoning situations.
[0025] For example, in NL2SQL tasks, Large Language Models (LLMs) often suffer from a lack of data modeling for "unknown fields / unknown tables" in their training data. This leads to the large model fabricating fields and / or tables that do not exist in the database during inference (i.e., "illusion"). For instance, if a user asks the large model "Statistics on customer age," even if the data to be queried in the database does not contain the age field, the large model may still generate the SQL statement "SELECT age FROM customers" and execute the query task. In other words, the large model may subsequently fabricate age data and provide an incorrect answer.
[0026] Generally, methods to suppress the large model illusion include data augmentation retrieval (RAG) and post-processing validation after model inference. These methods involve acquiring relevant knowledge from external data sources during model inference to improve the accuracy and contextual fit of AI-generated content; or performing data validation after model inference to identify and correct errors in the data generated by the large model.
[0027] However, current methods still cannot solve the illusion problem at its source. Even though RAG technology relies on external schema information to enhance the large model's learning of relevant knowledge, an incomplete schema can still lead to the large model generating incorrect SQL. Post-processing validation methods also increase the inference latency of the large model, and after incorrect answers have been generated, they can often only detect the errors, and it is difficult to correct answers with large deviations.
[0028] Please see Figure 1 , Figure 1 This is a flowchart illustrating an exemplary embodiment of the method for suppressing large model illusions according to this application. Specifically, it may include the following steps: Step S110: The obtained raw data to be queried is masked to obtain masked data.
[0029] It should be noted that in the example application of this application, an NL2SQL task can be executed through a large language model, and relevant data can be queried from the database based on the obtained SQL statement. Data can be stored in the database using a schema (equivalent to a data table). A schema is a collection used to define and organize data structures, and is also an abstract collection of metadata in the database.
[0030] Metadata refers to data that describes other data (such as basic information like data name, type, and format). In a database, the schema provides the original structural information, including table names, field names, field types, and field descriptions; this data is essentially metadata. Common data queries typically involve sending a query request containing metadata query conditions to locate the corresponding stored data in the database. Therefore, if metadata (such as table names and / or field names) is missing from the schema, it becomes difficult to find the corresponding stored data in the database.
[0031] The raw data to be queried is equivalent to the metadata (fields and tables) in the example above. Querying this metadata allows you to retrieve the stored data in the database. However, if this raw data is masked, it becomes impossible to query this metadata and retrieve the stored data in the database. When masking the raw data, you can mask all or part of the data; there is no limitation here. The following explanation will primarily focus on partial data masking.
[0032] For example, in a scenario where customer information is being queried, the original schema data table for customer information would look like this:
[0033] Here, "name" is the table name, "fields" are the fields in the table (which may include: customer ID "customer_id", customer name "name", and customer birthdate "birth_date"), "type" refers to the field type, and "label" refers to the pre-labeled information for the field (including required data "required" and optional data "optional"). Required data refers to data that must be retained, while optional data refers to data that can be hidden.
[0034] For the example schema above, the raw data (customer information) that users can query includes customer ID, customer name, and customer birthday. The customer birthday is optional, so it can be masked. The masked data is shown in the following example:
[0035] In other words, the "birth_date" field in the schema of the example above is masked using "masked_fields": ["birth_date"]. This is equivalent to the "birth_date" field not existing in the schema. If the larger model then receives a request for "birth_date" or similar requests requiring it, such as "calculate customer age," it will be unable to correctly find the stored data belonging to the "birth_date" field. In traditional methods, the larger model might subsequently fabricate customer "birth_date" data and provide incorrect answers, creating a hallucination problem.
[0036] Step S120: Generate negative question-and-answer pairs based on the masked data. The negative question-and-answer pairs are used to characterize the defects in the masked data.
[0037] To illustrate the preceding steps, large models trained using standard supervised learning in traditional methods are highly susceptible to the illusion problem when querying defective data. Therefore, this application trains a large model by actively constructing negative question-answer pairs involving defective data. This enables the large model to learn to refuse to provide direct, infeasible answers to questions involving defective data, rather than fabricating data or correcting defective or incorrect answers. Ultimately, the large model outputs negative answers (e.g., "The currently queried field is unavailable," "The currently queried field does not exist," "The currently queried information is unauthorized") to characterize the defect in the currently queried field (masked data).
[0038] For example, let's continue with the scenario where the customer's birthday is hidden in the schema of the example above. Based on the hidden data in the example above, a natural language (NL) question and negative answer can be generated, as shown in the following example:
[0039] In this context, the question in the negative question-and-answer pair is used to make a request regarding the blocked portion of the blocked data, while the answer in the negative question-and-answer pair is used to explicitly indicate that the current data is unavailable or does not exist when resolving the issue.
[0040] It should be noted that there are multiple methods for generating questions in this application, including generating negative question-answer pairs based on a preset question-answer template (designing question templates based on masked fields (such as "statistics...", "filtering...", etc.)), and / or generating negative question-answer pairs based on a large model (calling an LLM (such as GPT-4), providing the masked metadata and informing the generation rules to generate more natural questions), or manually setting negative question-answer pairs in advance, etc., which are not limited here.
[0041] Step S130: Adjust the obtained initial large model based on the negative question-answer pairs and masked data to obtain the target large model.
[0042] Based on the preceding steps, after obtaining the masked data and its corresponding negative question-answer pairs, the initial large model can be adjusted according to the negative question-answer pairs and the masked data to obtain the target large model. The initial large model can refer to either a pre-trained model or an untrained model; there is no limitation here.
[0043] For example, methods for adjusting the initial large model may include, but are not limited to, fine-tuning the model parameters of the initial large model by referring to model fine-tuning methods in the field; or continuing to train the initial large model by referring to model training methods in the field, etc., without limitation here. Ultimately, a well-adjusted target large model can be obtained. That is, in the subsequent application of the target large model, if a query involving defective data is received, a negative answer can be directly output (such as "the currently queried data is unavailable" or "the currently queried data does not exist"), thereby avoiding the large model illusion problem caused by the large model fabricating data to answer in traditional methods.
[0044] As can be seen, this application obtains raw data, such as from a database, and masks the table and / or field information of the raw data to obtain masked data with data defects. Based on the masked data, corresponding negative question-and-answer pairs are generated. These pairs include natural language questions that might be posed to the masked data and negative answers indicating that the masked data used to solve those questions is defective. The initial large model (such as an NL2SQL generative model) obtained from the negative question-and-answer pairs and the masked data is trained and adjusted to obtain the target large model. This enables the target large model to output negative answers (such as indicating that certain data in the query is unavailable or does not exist) when receiving queries containing defective data in practical applications, instead of generating positive answers to solve queries as is the case with traditional generative large models. This avoids the illusion problem that generative large models may produce during data generation.
[0045] Based on the above embodiments, this application embodiment exemplarily describes the steps of masking the obtained original data to be queried to obtain masked data. Specifically, the method of this embodiment includes the following steps: Obtain the pre-labeled information of the raw data, which includes required data and / or optional data; mask the optional data in the raw data to obtain masked data.
[0046] Referring to the foregoing embodiments, before masking the original data, it is necessary to pre-label the original data to obtain pre-labeled information for each piece of original data. This pre-labeling information can be based on whether each piece of data is required or optional according to its necessity in its business scenario (application scenario), and then labeled accordingly. Required data refers to data that is essential for implementing a certain data service.
[0047] It should be noted that the data pre-labeling process can be based on methods such as automatic labeling using a pre-defined rule engine or manual labeling; no limitation is made here. For example, the process implemented based on a rule engine can be automatic labeling based on predefined labeling rules (such as field names containing id, name, etc., or rules defined in the database schema as primary key, unique key, NOT NULL, etc.). After automatic labeling, manual verification can be performed, allowing manual review of the labels (pre-labeled information) of key fields through labeling tools.
[0048] Furthermore, after the data pre-labeling is completed, the labeled schema (including the pre-labeling information of each original data) can be obtained. This pre-labeling information can characterize whether each original data is required or optional. The optional data is then masked to obtain masked data. The process of masking optional data can involve masking all or part of the optional data in the original data; this is not limited here.
[0049] Generally speaking, a schema may include both required and optional data. Therefore, by masking all or part of the optional data (such as fields / tables), a partially visible schema (masked data) can be obtained. It can also record which data is masked so that negative question-and-answer pairs can be generated later based on the masked data.
[0050] Based on the above embodiments, this application embodiment exemplarily describes the steps of masking optional data in the original data to obtain masked data. Specifically, the method of this embodiment includes the following steps: Obtain the preset masking ratio; based on the preset masking ratio, mask the table information and / or field information of the target data in the optional data to obtain the masked data.
[0051] In conjunction with the foregoing embodiments, when masking optional data in this application, a portion of the optional data may be masked according to a preset masking ratio.
[0052] For example, a preset blocking ratio (e.g., 30%) can be obtained, and data marked as "optional data" in the schema field information and / or table information can be randomly blocked according to the preset blocking ratio. Alternatively, selective blocking can be performed. Specific blocking rules can be defined according to the actual application scenario (e.g., access control for data access, data isolation, etc.), which are not limited here.
[0053] Based on the above embodiments, this application embodiment describes the steps of masking table information and / or field information of target data in optional data according to a preset masking ratio to obtain masked data. Specifically, the method of this embodiment includes the following steps: Obtain the associated data of the target data; mask the table information and / or field information of the target data, as well as the table information and / or field information of the associated data, to obtain masked data.
[0054] Referring to the foregoing embodiments, multiple data items in a database may have certain relationships. For example, two or more fields may be used together to express a complete business logic in a real-world application scenario, typically requiring these data to either both exist or neither exists. Therefore, when masking target data with these relationships, it is necessary to mask their related data as well, thus obtaining the masked data.
[0055] For example, the data masking process for vehicle query scenarios can be exemplified as follows:
[0056] Therefore, for applications targeting vehicle violation queries, hphm and hpzl are related data and should ideally both exist. For applications targeting pedestrian violation queries, both hphm and hpzl can be disabled.
[0057] Based on the above embodiments, this application embodiment describes the steps of masking table information and / or field information of target data in optional data according to a preset masking ratio to obtain masked data. Specifically, the method of this embodiment includes the following steps: Obtain the data importance of the optional data; determine the target data in the optional data based on the data importance; mask the table information and / or field information of the target data to obtain masked data.
[0058] In conjunction with the foregoing embodiments, the process of masking optional data can be either partially or randomly masked according to a preset masking ratio, or selectively masked according to a preset masking ratio.
[0059] For example, optional data (such as fields) can be obtained by sorting the optional data from high to low importance. (The higher the importance of the data, the more it needs to be masked, so that the large model after fine-tuning training will refuse to generate positive answers to questions when important data is lacking in actual applications.)
[0060] Methods for determining data importance may include, but are not limited to, obtaining pre-set data importance for each piece of data, or determining the corresponding data importance based on the usage frequency, business weight, etc. For example, data importance can be calculated based on usage frequency and / or business weight using a pre-defined data importance calculation function. Usage frequency and business weight are positively correlated with data importance.
[0061] Based on the above embodiments, this application embodiment describes the steps for generating negative question-and-answer pairs based on masking data. Specifically, the method of this embodiment includes the following steps: Based on the masked data, natural language questions and their corresponding negative answers are generated. The negative answers indicate that the masked data has defects when used to solve natural language questions. The natural language questions and their negative answers are identified as negative question-answer pairs.
[0062] Referring to the foregoing embodiments, after obtaining the partially visible schema (masked data), the masked data can be input into the question-answer generation module to generate a natural language question and its corresponding negative answer.
[0063] For example, the process of generating questions can be done through a template-based approach, such as designing question templates (e.g., "statistics...", "filtering...") based on masked data (masked fields) to generate questions. Alternatively, questions can be generated by generating a large model (which could be a large model to be fine-tuned in this application or another large model). For example, by calling an LLM (e.g., GPT-4), providing the LLM with masked data, and informing it of the generation rules, more natural questions can be generated.
[0064] Negative answers, on the other hand, can characterize the deficiencies in the masked data when used to solve natural language problems, enabling the fine-tuned large model to provide negative responses (explicitly pointing out problems such as the absence or unavailability of fields / tables, etc.). Therefore, the natural language questions and negative answers generated based on each masked data can be identified as the negative question-answer pairs corresponding to that masked data.
[0065] For example, after at least disabling the hphm and hpzl fields in the aforementioned example, negative question-answer pairs can be generated based on the disabled data:
[0066] Based on the above embodiments, this application embodiment describes the steps of adjusting the obtained initial large model according to negative question-answer pairs and masking data to obtain the target large model. Specifically, the method of this embodiment includes the following steps: Multiple masked data and their corresponding negative question-answer pairs are organized into a negative sample set; the initial large model is then fine-tuned based on the negative sample set to obtain the target large model.
[0067] Referring to the foregoing embodiments, the method described above can mask multiple fields and / or tables, resulting in various masked data. Corresponding negative question-answer pairs are then generated based on each type of masked data. Each type of masked data (local metadata) and its corresponding generated questions and negative answers are packaged into training samples (negative samples). The training samples corresponding to multiple masked data are then organized into a negative sample set (including multiple negative samples). The initial large model can then be fine-tuned based on the negative sample set to obtain the target large model.
[0068] For example, reference can be made to, such as Figure 2 As shown, Figure 2 This is an exemplary overall process diagram of a method for suppressing large model illusions according to this application.
[0069] As can be seen from the above, the overall implementation process of this application may include at least the following steps: 1. Input the raw data (raw metadata) of the database: This may include all tables in the database, as well as the field names, field types, field meanings and descriptions, and field relationships of each table.
[0070] 2. Metadata Annotation: Label tables and fields in the database schema as "required data" or "optional data." Required data typically consists of core business fields (such as `customer_id`), and disabling these fields prevents the answering of related questions. Optional fields are usually auxiliary business fields (such as `birth_date`), and disabling them requires the larger model to determine whether a response is possible based on the actual application scenario (e.g., it cannot answer questions related to birthdays, but can answer questions unrelated to birthdays).
[0071] 3. Metadata Masking: Mask "optional" fields / tables based on preset strategies (such as random probability, business rules) to generate a partial metadata view. You can also choose to mask some "essential" fields of the original data, etc., without limitation here. For example: mask the field birth_date, and retain customer_id, etc.
[0072] 4. Generate negative question-and-answer pairs based on the masked data.
[0073] 5. Output the training dataset.
[0074] For specific fine-tuning training methods, please refer to relevant model fine-tuning techniques in this technical field; no specific limitations are made here.
[0075] Based on the above embodiments, this application embodiment describes the steps of adjusting the obtained initial large model according to negative question-answer pairs and masking data to obtain the target large model. Specifically, the method of this embodiment includes the following steps: Multiple masked data and their corresponding negative question-answer pairs are organized into a negative sample set; the positive sample set corresponding to the original data is obtained; the initial large model is fine-tuned based on the negative sample set and the positive sample set to obtain the target large model.
[0076] Referring to the foregoing embodiments, in this application, when fine-tuning the initial large model, a negative sample set constructed based on masked data and its corresponding negative question-and-answer pairs can be used as part of the training set. Furthermore, a positive sample set corresponding to the original data can be introduced as part of the training set. That is, the initial large model is fine-tuned based on the negative and positive sample sets to obtain the target large model.
[0077] In this context, the positive sample set refers to the set generated by using unmasked normal data (non-defective data) in common model training methods, and then packaging the normal data and its question-answer pairs together as training samples.
[0078] In summary, the overall system architecture of this application can be referenced as follows: Figure 3 As shown, Figure 3 This is an exemplary system architecture diagram of a method for suppressing large model illusions according to this application.
[0079] The example system architecture described above includes: 1. Data Input Layer: 1.1 The raw database schema provides the original structure information of the database, including table names, field names, field types, and field descriptions. The input format is a database metadata file (such as an SQL script or a JSON-formatted schema). It primarily serves as the starting point for the entire system, providing foundational data for subsequent annotation and masking strategies.
[0080] The 1.2 metadata annotation module can automatically or manually annotate fields and tables in the database schema with "required" or "optional" tags, and determine the relationships between fields (for example, two or more fields work together to express a complete business logic, which usually requires them to exist together or not exist at all).
[0081] 2. Data Processing Layer: 2.1 The metadata masking engine can dynamically mask optional data based on the pre-annotated schema, generate locally visible schemas, and record which schemas are masked so as to generate targeted negative question-and-answer pairs.
[0082] 2.2 The question-answer generation module can generate natural language questions and negative answers based on information such as masked fields / tables.
[0083] 3. Data Output Layer: 3.1 The training data generation module can package the local schema and its corresponding questions and negative answers into standardized training samples.
[0084] 3.2 Training dataset, which can output a complete training dataset for fine-tuning the initial large model.
[0085] The method for suppressing large-model hallucinations provided in this application, compared with traditional methods, is as follows: 1. Negative samples with data defects such as "missing fields" and "missing tables" can be injected from the training data source to solve the illusion problem from the source of SQL generation, without relying on the post-processing of SQL generated by the large model.
[0086] 2. By labeling whether data (such as fields) is optional and the data relationships, we can ensure that even dynamically generated data tables with "missing fields" can still solve basic business problems and have business practicality.
[0087] 3. By randomly constructing tables and negative samples with missing fields, we can ensure the diversity and randomness of the samples and improve the training effect.
[0088] 4. It can not only simulate missing tables and fields in physical logic, but also supports complex scenarios such as permission simulation (e.g., blocking data that does not have access permissions).
[0089] 5. Build a lightweight schema by semantically annotating metadata (such as field "necessity" tags and relationships) to reduce reliance on large-scale annotated data.
[0090] 6. It has cross-domain versatility, and its core technology can be adapted to various tasks (such as NL2API and NL2Chart), not just NL2SQL, to meet different business needs.
[0091] 7. It has a low dependence on the large model itself. When generating samples, the large model, which theoretically has a certain simultaneous capability, can synthesize the problem well with the input basic metadata background.
[0092] It should be further noted that the entity executing the method for suppressing large model illusions can be a device for suppressing large model illusions. For example, the method for suppressing large model illusions can be executed by a terminal device, a server, or other processing devices. The terminal device can be a user equipment (UE), computer, mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. In some possible implementations, the method for suppressing large model illusions can be implemented by a processor calling computer-readable instructions stored in memory.
[0093] Figure 4 This is a block diagram illustrating a device for suppressing large-model illusions, as shown in an exemplary embodiment of this application. Figure 4 As shown, the exemplary device 400 for suppressing large model illusion includes: a data masking module 410, a question-and-answer generation module 420, and a model adjustment module 430. Specifically: The data masking module 410 is used to mask the acquired raw data to be queried, and obtain masked data.
[0094] The question-and-answer generation module 420 is used to generate negative question-and-answer pairs based on the masked data. The negative question-and-answer pairs are used to characterize the defects in the masked data.
[0095] The model adjustment module 430 is used to adjust the obtained initial large model based on negative question-answer pairs and masked data to obtain the target large model.
[0096] In this exemplary device for suppressing the illusion of large models, raw data, such as data from a database, is acquired, and the table and / or field information of the raw data is masked to obtain masked data with data defects. Corresponding negative question-and-answer pairs are generated based on the masked data. These pairs include a natural language question that might be posed to the masked data and a negative answer indicating that the masked data used to solve the natural language question is defective. The initial large model (such as an NL2SQL generative large model) obtained based on the negative question-and-answer pairs and the masked data is trained and adjusted to obtain the target large model. This enables the target large model to output a negative answer (such as indicating that certain data in the query is unavailable or does not exist) when receiving a query containing defective data in practical applications, instead of generating a positive answer to solve the query as traditional generative large models always do. This avoids the illusion problem that generative large models may generate during data generation from the source.
[0097] It should be noted that the apparatus and method provided in the above embodiments belong to the same concept, and the specific ways in which each module and unit performs operations have been described in detail in the method embodiments, and will not be repeated here. In practical applications, the apparatus provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the apparatus can be divided into different functional modules to complete all or part of the functions described above, and this is not a limitation.
[0098] The functions of each module can be found in the implementation examples of the method for suppressing large model illusions, which will not be repeated here.
[0099] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of an embodiment of the electronic device of this application. The electronic device 100 includes a memory 101 and a processor 102. The processor 102 is used to execute program instructions stored in the memory 101 to implement the steps in any of the above-described method embodiments for suppressing large model illusions. In a specific implementation scenario, the electronic device 100 may include, but is not limited to, a microcomputer or a server. In addition, the electronic device 100 may also include mobile devices such as laptops and tablets, which are not limited here.
[0100] Specifically, processor 102 controls itself and memory 101 to implement the steps in any of the above-described method embodiments for suppressing large model illusions. Processor 102 may also be referred to as a CPU (Central Processing Unit). Processor 102 may be an integrated circuit chip with signal processing capabilities. Processor 102 may also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component. A general-purpose processor may be a microprocessor or any conventional processor. Furthermore, processor 102 may be implemented using integrated circuit chips.
[0101] In this exemplary electronic device, raw data, such as data from a database, is acquired, and the table and / or field information of the raw data is masked to obtain masked data with data defects. Corresponding negative question-and-answer pairs are generated based on the masked data. These pairs include a natural language question that might be posed to the masked data and a negative answer indicating that the masked data used to solve the natural language question is defective. The initial large model (such as an NL2SQL generative large model) obtained from the negative question-and-answer pairs and the masked data is trained and adjusted to obtain a target large model. This enables the target large model to output a negative answer (such as indicating that certain data in the query is unavailable or does not exist) when receiving a query containing defective data in practical applications, instead of generating a positive answer to solve the query as is the case with traditional generative large models. This avoids the illusion problem that generative large models may produce during data generation from the source.
[0102] Please see Figure 6 , Figure 6 This is a schematic diagram of a computer-readable storage medium according to an embodiment of the present application. The computer-readable storage medium 110 stores program instructions 111 that can be executed by a processor. The program instructions 111 are used to implement the steps in any of the above-described method embodiments for suppressing large model illusions.
[0103] In this exemplary storage medium, program instructions within the storage medium are executed to obtain raw data, such as data from a database. Table and / or field information of the raw data is then masked to obtain masked data with data defects. Corresponding negative question-and-answer pairs are generated based on the masked data. These pairs include natural language questions that might be posed to the masked data and negative answers indicating the defects in the masked data used to solve those questions. The initial large model (such as an NL2SQL generative model) obtained from the negative question-and-answer pairs and the masked data is trained and adjusted to obtain the target large model. This enables the target large model to output negative answers (such as indicating that certain data in the query is unavailable or does not exist) when receiving queries containing defective data in practical applications, instead of generating affirmative answers to solve queries as is the case with traditional generative large models. This avoids the illusion problem that generative large models may produce during data generation.
[0104] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.
[0105] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.
[0106] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.
[0107] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
Claims
1. A method for suppressing large-scale hallucinations, characterized in that, The method includes: The obtained raw data to be queried is masked to obtain masked data; Based on the masked data, a negative question-and-answer pair is generated, which is used to characterize the existence of defects in the masked data; The initial large model obtained is adjusted based on the negative question-answer pairs and the masked data to obtain the target large model.
2. The method according to claim 1, characterized in that, The process of masking the obtained raw data to be queried to obtain masked data includes: Obtain the pre-labeling information of the raw data, wherein the pre-labeling information includes required data and / or optional data; The optional data in the original data is masked to obtain the masked data.
3. The method according to claim 2, characterized in that, The process of masking optional data in the original data to obtain the masked data includes: Get the preset blocking ratio; The table information and / or field information of the target data in the optional data are masked according to the preset masking ratio to obtain the masked data.
4. The method according to claim 3, characterized in that, The step of masking the table information and / or field information of the target data in the optional data according to the preset masking ratio to obtain the masked data includes: Obtain the associated data of the target data; The table information and / or field information of the target data, as well as the table information and / or field information of the associated data, are masked to obtain the masked data.
5. The method according to claim 3, characterized in that, The step of masking the table information and / or field information of the target data in the optional data according to the preset masking ratio to obtain the masked data includes: Obtain the data importance of the optional data; The target data in the optional data is determined based on the importance of the data. The table information and / or field information of the target data are masked to obtain the masked data.
6. The method according to claim 1, characterized in that, The step of generating negative question-answer pairs based on the masked data includes: A natural language question and a corresponding negative answer are generated based on the masked data, wherein the negative answer indicates that the masked data has a defect in solving the natural language question; The natural language question and the negative answer are identified as the negative question-answer pair.
7. The method according to claim 1, characterized in that, The step of adjusting the obtained initial large model based on the negative question-answer pairs and the masking data to obtain the target large model includes: Multiple masked data and their corresponding negative question-and-answer pairs are organized into a negative sample set; The initial large model is fine-tuned based on the negative sample set to obtain the target large model.
8. The method according to claim 1, characterized in that, The step of adjusting the obtained initial large model based on the negative question-answer pairs and the masking data to obtain the target large model includes: Multiple masked data and their corresponding negative question-and-answer pairs are organized into a negative sample set; Obtain the positive sample set corresponding to the original data; The initial large model is fine-tuned based on the negative sample set and the positive sample set to obtain the target large model.
9. An electronic device, characterized in that, The method includes a memory and a processor, the processor being configured to execute program instructions stored in the memory to implement the method according to any one of claims 1 to 8.
10. A computer-readable storage medium having program instructions stored thereon, characterized in that, When the program instructions are executed by the processor, they implement the method described in any one of claims 1 to 8.