Sample generation method and device based on large model and agent

Abstract

The present disclosure provides a large model-based sample generation method and device and an agent, and relates to the technical field of artificial intelligence, in particular to the technical field of large language models, AI intelligent assistants and machine learning. The specific implementation scheme is as follows: receiving at least one question selected by an object on a question analysis interface; the at least one question is obtained by analyzing the reason of an abnormal query result output by a retrieval model; and based on the at least one question, a large model is used to modify a target query sentence and a target query result to generate a target sample, so as to optimize the retrieval model for the at least one question based on the target sample; wherein the target query sentence is associated with an initial query sentence in the abnormal query result; and the target query result is obtained by querying the target query sentence based on the retrieval model.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to the fields of large language models, AI intelligent assistants, and machine learning technology, specifically to sample generation methods, devices, intelligent agents, electronic devices, storage media, and program products based on large models. Background Technology

[0002] In the field of machine learning technology, for retrieval models, random sampling or sampling based on sampling rules is usually used to increase the number of training samples in order to optimize the search model.

[0003] However, increasing the number of training samples by using random sampling or sampling based on sampling rules makes it difficult to optimize the retrieval model and meet users' performance requirements for the retrieval model. Summary of the Invention

[0004] This disclosure provides a method, apparatus, intelligent agent, electronic device, storage medium, and program product for generating samples based on a large model.

[0005] According to one aspect of this disclosure, a sample generation method based on a large model is provided, comprising: receiving an object selecting at least one question on a question analysis interface; the at least one question being obtained by performing causal analysis on abnormal query results output by a retrieval model; and based on the at least one question, modifying a target query statement and a target query result using a large model to generate a target sample, so as to optimize the retrieval model for the at least one question based on the target sample; wherein the target query statement is associated with an initial query statement in the abnormal query results; and the target query result is obtained by querying based on the target query statement using a retrieval model.

[0006] According to another aspect of this disclosure, a sample generation apparatus based on a large model is provided, comprising: a receiving module for receiving at least one question selected by an object on a question analysis interface; the at least one question is obtained by performing causal analysis on abnormal query results output by a retrieval model; and a generation module for modifying a target query statement and a target query result based on the at least one question using a large model to generate a target sample, so as to optimize the retrieval model for the at least one question based on the target sample; wherein the target query statement is associated with an initial query statement in the abnormal query results; and the target query result is obtained by querying based on the target query statement using the retrieval model.

[0007] According to another aspect of this disclosure, an intelligent agent for sample generation is provided, comprising: an input module for receiving at least one question selected by an object on a question analysis interface; a processing module for determining a target task based on the at least one question information received by the input module, determining a target large model based on the target task, and obtaining a target sample for optimizing a retrieval model by calling the target large model to execute the above method; and an output module for outputting the target sample obtained by the processing module.

[0008] According to another aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method described above.

[0009] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause a computer to perform the methods described above.

[0010] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the method described above.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0012] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0013] Figure 1 The illustration schematically shows an exemplary system architecture for applying large-model-based sample generation methods and apparatus according to embodiments of the present disclosure;

[0014] Figure 2 A flowchart illustrating a large-model-based sample generation method according to an embodiment of the present disclosure is shown schematically.

[0015] Figure 3 This illustration schematically shows a diagram of the generation of a target query statement according to an embodiment of the present disclosure;

[0016] Figure 4 A schematic diagram illustrating the determination of a target strategy according to an embodiment of this disclosure is shown.

[0017] Figure 5 The illustration shows a schematic diagram of a specific target strategy example according to an embodiment of the present disclosure;

[0018] Figure 6 A schematic diagram illustrating the generation of a target sample according to an embodiment of the present disclosure is shown.

[0019] Figure 7 This illustration schematically shows a diagram of target text generation according to an embodiment of the present disclosure;

[0020] Figure 8 A schematic diagram illustrating a design based on an embodiment of this disclosure is shown.

[0021] Figure 9 A schematic diagram illustrating the generation of a target sample according to another embodiment of this disclosure is shown.

[0022] Figure 10 A schematic diagram illustrating the generation of training samples according to embodiments of the present disclosure is shown.

[0023] Figure 11 A block diagram of a large-model-based sample generation apparatus according to an embodiment of the present disclosure is shown schematically.

[0024] Figure 12 A block diagram of an intelligent agent for sample generation according to embodiments of the present disclosure is schematically shown; and

[0025] Figure 13 A block diagram of an electronic device suitable for implementing a large model-based sample generation method according to an embodiment of the present disclosure is shown schematically. Detailed Implementation

[0026] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0027] For retrieval models, training samples are increased using random sampling or sampling rule-based sampling. However, because retrieval models cannot accurately target specific abnormal samples when using random sampling or sampling rule-based sampling, the accuracy of the retrieval model in meeting user query needs is insufficient, affecting the user search experience and the quality of the retrieval model. For example, there may be a mismatch between the query statement and the query results for abnormal samples.

[0028] Although abnormal samples can be manually labeled, manual labeling is inefficient and difficult to cover a wide range of abnormal samples.

[0029] Furthermore, although performance can be improved by refining the model architecture of the retrieval model, the improved retrieval model still lacks the ability to perform targeted optimization for complex semantic understanding problems.

[0030] In view of this, the embodiments of this disclosure address the problem of abnormal query results output by the retrieval model by using a large model to modify the target query statement associated with the initial query statement in the abnormal query results and the target query results obtained by the retrieval model, thereby generating training samples in a targeted manner. Since the training samples are obtained through targeted modifications to address the problem of abnormal query results output by the retrieval model, optimizing the retrieval model based on these training samples can improve the retrieval model's ability to identify abnormal query results, enhance the accuracy of the query results output by the retrieval model, and improve the user experience.

[0031] Figure 1 The illustration schematically shows an exemplary system architecture for applying large-model-based sample generation methods and apparatus according to embodiments of the present disclosure.

[0032] It is important to note that Figure 1 The examples shown are merely examples of system architectures that can be applied to embodiments of this disclosure, to help those skilled in the art understand the technical content of this disclosure, but do not imply that embodiments of this disclosure cannot be used in other devices, systems, environments, or scenarios. For example, in another embodiment, an exemplary system architecture for applying the large model-based sample generation method and apparatus may include a terminal device, but the terminal device may implement the large model-based sample generation method and apparatus provided by embodiments of this disclosure without interacting with a server.

[0033] like Figure 1 As shown, the exemplary architecture 100 may include a terminal device 101, an intelligent agent 102, a server 103, and a model library 104.

[0034] Various communication client applications can be installed on the terminal device 101, such as knowledge reading applications, web browser applications, search applications, instant messaging tools, email clients and / or social platform software, AI smart assistants, etc. (for example only).

[0035] Terminal device 101 can be various electronic devices with a display screen and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0036] Agent 102 can be based on large models, such as large language models, to identify user needs and output information that meets those needs.

[0037] The model library 104 may include, for example, multiple models trained based on various deep learning algorithms.

[0038] Server 103 can be a server that provides various services, such as a backend management server that supports the content browsed by the user using terminal device 101 (for example only). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to terminal device 101.

[0039] For example, an object can input at least one question selected on the question analysis interface into terminal device 101. Terminal device 101 can invoke agent 102 to determine the target task based on the at least one question, and based on the target task, call the target large model from model library 104. The target large model is then used to modify the target query statement and query results to generate target text. Then, the target text generated by the target large model is used to optimize the search model to solve the aforementioned problem of not being able to meet the user's performance requirements for the search model.

[0040] It should be noted that the large model-based sample generation method provided in this disclosure can generally be executed by the terminal device 101. Accordingly, the large model-based sample generation device provided in this disclosure can also be located in the terminal device 101.

[0041] Alternatively, the large-model-based sample generation method provided in this disclosure can generally be executed by server 103. Correspondingly, the large-model-based sample generation apparatus provided in this disclosure can generally be located in server 103. The large-model-based sample generation method provided in this disclosure can also be executed by a server or server cluster that is different from server 103 and capable of communicating with terminal device 101 and / or server 103. Correspondingly, the large-model-based sample generation apparatus provided in this disclosure can also be located in a server or server cluster that is different from server 103 and capable of communicating with terminal device 101 and / or server 103.

[0042] For example, terminal device 101 can send at least one question selected by the user on the question analysis interface to server 103. After receiving at least one question, server 103 can execute the large model-based sample generation method of this disclosure embodiment to call agent 102 to generate target samples. Then, the target samples are fed back to terminal device 101 so that the device can use the target samples to optimize the model.

[0043] In the technical solution disclosed herein, the collection, storage, use, processing, transmission, provision, disclosure, and application of user personal information comply with the provisions of relevant laws and regulations, necessary confidentiality measures have been taken, and there is no violation of public order and good morals.

[0044] In the technical solution disclosed herein, the user's authorization or consent is obtained before acquiring or collecting the user's personal information.

[0045] Figure 2 A flowchart illustrating a large-model-based sample generation method according to an embodiment of the present disclosure is shown schematically.

[0046] like Figure 2 As shown, the method 200 includes operations S210~S220.

[0047] In operation S210, the receiving object selects at least one problem on the problem analysis interface.

[0048] In operation S220, based on at least one question, the target query statement and target query results are modified using a large model to generate target samples, so that the retrieval model can be optimized for at least one question based on the target samples.

[0049] In this embodiment of the disclosure, at least one problem is obtained by analyzing the causes of abnormal query results output by the retrieval model.

[0050] For example, the object may include, but is not limited to, users who train or optimize the retrieval model. The problem analysis interface may be an interactive interface used by the intelligent agent to display the analysis results. The retrieval model may be any model trained based on machine learning, deep learning, or other methods.

[0051] At least one problem may include, but is not limited to, issues such as missing keywords, abnormal entity matching, abnormal word order, text errors, abnormal keyword importance, and abnormal synonyms.

[0052] Keywords may include at least one of the following: words in the query statement, words in the text title, important words in the text summary, important words in the text paragraph, etc.

[0053] Entities may include at least one of the following: personal names, place names, organizations, subject entities, etc. in the text.

[0054] Word order disorder can include at least one of the following: semantic differences or semantic similarities caused by disordered word order in the text.

[0055] Text errors may include at least one of the following: text that does not conform to language norms, text with logical errors, text with inaccurate information, etc. Keyword importance may include, for example, the importance of keywords to the search requirements. Synonym anomalies may include, for example, unidentified or associated words with the same or similar meanings as words in the text.

[0056] For example, an anomalous query result indicates that at least one text output by the model does not match the initial query statement. For instance, if the initial query statement is "Can physical cooling methods be used to reduce fever in children?", an anomalous query result could be at least one of the following: "Text about physical cooling methods for adults with fever", "Text about discussing the health effects of colds on children", "Text about drug treatment methods for fever in children", etc.

[0057] For example, for an abnormal query result like "text about physical cooling methods for adults with fever," the agent calls a large model to perform cause analysis and outputs an indication of missing keywords. For an abnormal query result like "text about discussing the health effects of colds on children," the agent calls a large model, such as a large language model, and uses its deep understanding capabilities to perform cause analysis of the abnormal query result, outputting an indication of grammatical order errors. For an abnormal query result like "text about drug treatment methods for children with fever," the agent outputs an indication of subject entity mismatch.

[0058] In this embodiment of the disclosure, the target query statement is associated with the initial query statement in the abnormal query results. The target query results are obtained using a retrieval model based on the target query statement.

[0059] For example, target query statements with similar semantic features can be determined based on the semantic features of the initial query statement. The target query statement is then input into the retrieval model, and the target query results are output.

[0060] For example, if the initial query is "Can physical cooling methods be used to reduce a child's fever?", the target query could be "Can physical cooling methods be used to reduce a child's fever?" and / or "Can physical cooling methods be used to reduce a child's fever?" and / or "Can a wet towel be used to reduce a child's fever?". The target query results could be "text about physical cooling methods for children's fever" and / or "text about using wet towels to reduce a child's fever".

[0061] For example, to address the issue of missing indicator keywords, the query can be modified to change "text about physical cooling for children's fever" to "text about physical cooling for adults' fever" for the keyword "children" in the target query such as "Can children's fever be cooled down using physical methods?".

[0062] This disclosure addresses the problem of abnormal query results output by the retrieval model by using a large model to modify the target query statement associated with the initial query statement in the abnormal query results and the target query results obtained by the retrieval model, thereby generating targeted training samples. Since the training samples are obtained through targeted modifications to address the problem causing abnormal query results output by the retrieval model, optimizing the retrieval model based on these training samples can improve the retrieval model's ability to identify abnormal query results, enhance the accuracy of the retrieval model's output query results, and improve the user experience.

[0063] The following is for reference. Figures 3-10 In conjunction with specific embodiments, Figure 2 The method shown will be further explained.

[0064] In this embodiment of the disclosure, the cause analysis of abnormal query results output by the retrieval model may specifically include the following operations: generating a question prompt based on the abnormal query results and the initial query statement of the abnormal query results, inputting the question prompt into the intelligent agent, and the intelligent agent calling the large language model to output at least one question.

[0065] By providing prompts, large language models can better understand the task requirements and obtain more accurate prompts.

[0066] In some embodiments of this disclosure, in order to improve the retrieval model's ability to identify similar query statements, the sample generation method based on the large model, in addition to... Figure 2 In addition to the operations S210~S220 shown, the operation also includes: enhancing the content of the initial query statement in the abnormal query results to generate the target query statement.

[0067] For example, content enhancement may include, but is not limited to: enhanced logic, enhanced detail, generalization of synonyms, generalization within the same domain, and generalization across domains.

[0068] Synonym generalization can include maintaining complete semantic elements while changing the way they are expressed.

[0069] Domain-specific generalization can include expanding query scenarios, query roles, query times, spatial dimensions, etc., within the same domain.

[0070] Cross-domain generalization can include extending queries to other domains through structure mapping.

[0071] By enhancing the content of the initial query, the retrieval model's ability to identify similar queries can be improved.

[0072] Figure 3 A schematic diagram illustrating the generation of a target query statement according to an embodiment of the present disclosure is shown.

[0073] like Figure 3 As shown, in this embodiment, content enhancement is performed on the initial query statement in the abnormal query results to generate a target query statement. This may include the following operations: determining the query element 302 in the initial query statement 301; and using the large model M to enhance the content of the initial query statement 301 according to the query element 302 to generate the target query statement 303.

[0074] For example, query elements are used to indicate key conditions used to locate and filter query results in the initial query statement 301.

[0075] For example, when the initial query is "Can physical cooling methods be used to reduce fever in children?", the query elements can be "children", "fever", "physical cooling", etc.

[0076] When the search term is "children," the content is enhanced to include "children, infants," etc. When the search term is "fever," the content is enhanced to include "fever, elevated body temperature, high fever," etc. When the search term is "physical cooling," the content is enhanced to include "warm water sponging, ice packs," etc.

[0077] By identifying the query elements of the initial query statement and using a large model to enhance the content of the query elements, the target query statement can maintain similar semantic features to the initial query statement, thereby achieving generalization of the initial query statement and improving the coverage of samples in similar queries, thus enhancing the model's generalization ability.

[0078] For example, query elements may include at least one of the following: query scenario, query role, query dimension, query domain, etc.

[0079] For example, query elements can include query scenarios. Based on the query elements, the initial query statement is enhanced using a large model to generate a target query statement. This can include the following operation: based on the query scenario, the initial query statement's query scenario is expanded using a large model to generate the target query statement.

[0080] The query context can be used to indicate the specific situation in which the query is performed.

[0081] Expanding query scenarios can include replacing them with scenarios of the same category but different specific entities, for example, expanding "Brand A mobile phone repair" to "Brand B mobile phone repair". Expanding query scenarios can also include adjusting the specific parameters of the limiting conditions, for example, expanding "mobile phones under 3000 yuan" to "mobile phones under 2000 yuan". Expanding query scenarios can also include replacing them with requirements that have the same function but different expressions, for example, expanding "weight loss recipes" to "calorie-controlled diet plans".

[0082] By expanding the query scenarios, more queries with similar semantic features to the initial query statement can be generated, thereby increasing the coverage of specific problem types from the query scenarios.

[0083] For example, query elements can also include query roles. Based on the query elements, the initial query statement is enhanced using a large model to generate the target query statement. This can also include the operation of expanding the query roles of the initial query statement using a large model to generate the target query statement.

[0084] Query roles can be used to indicate the entities participating in the query process. Different query domains have different query roles. Extending query roles can involve changing the query role within a query domain type. For example, in the professional consulting category, the specific consulting object can be changed within professional consulting fields such as medicine, law, and education; in the goods and services category, the specific goods or services can be changed within e-commerce and local services; in the content consumption category, the specific content type can be changed within media and entertainment; in the tool query category, the specific query type can be changed within information query and online tool categories; and in the life services category, the specific service type can be changed within transportation and utility payment categories.

[0085] By further expanding the query roles, more queries with similar semantic features to the initial query can be generated, enhancing content enhancement capabilities and increasing the coverage of specific question types from the query roles.

[0086] For example, query elements can also include query dimensions. Based on the query elements, the initial query statement is enhanced using a large model to generate the target query statement. This can also include the operation of expanding the query dimensions of the initial query statement using a large model to generate the target query statement.

[0087] Query dimensions can include query time and query space dimensions. For example, "the latest products of a certain brand now" can be expanded to "the latest products of a certain brand at the current query time"; "nearby restaurants" can be expanded to "restaurants near the user's location", etc.

[0088] By further expanding the query time and space dimensions, the coverage of specific question types is improved from the query time and space dimensions.

[0089] For example, query elements can also include query domains. Based on the query elements, the initial query statement is enhanced using a larger model to generate a target query statement. This can include operations such as expanding the query domain of the initial query statement using a larger model to generate the target query statement.

[0090] Extending the query domain can include expanding the query to different domains through structural mapping and element migration. Structural mapping involves generalizing according to the semantic structure of the initial query and the relationships between its elements to ensure that the expanded target query retains all the query elements from the initial query. Element migration involves converting elements such as the subject matter, constraints, requirements, and scenarios of the initial query into corresponding entities, constraints, demands, and scenarios across different domains.

[0091] For example, the field of medical consultation has expanded into the field of legal resources. The field of product purchase has expanded into the field of service booking. The field of education and training has expanded into the field of skills learning. The field of job seeking and career selection has expanded into the field of school selection, and so on.

[0092] By further expanding the query domain, the coverage of specific question types can be improved from the query domain.

[0093] Therefore, the initial query statement can be diversified and expanded by various query elements in the initial query statement, thereby generalizing the initial query statement. The resulting multiple target query statements can cover the query situation of specific question types, and realize the retrieval model's ability to identify query statements of the same type.

[0094] According to embodiments of this disclosure, regarding the above... Figure 2 The operation S220 shown, based on at least one question, modifies the target query statement and target query results using a large model to generate target samples, so as to optimize the retrieval model for at least one question based on the target samples. It may include the following operations: determining a target strategy based on at least one question; and modifying the target query statement and target query results using a large model based on the target strategy to generate target samples.

[0095] In this embodiment of the disclosure, the target strategy indicates the distinguishing factors between positive and negative samples used to optimize the retrieval model.

[0096] For example, positive samples can include text and query statements that satisfy the query requirements from multiple texts retrieved using the query statement. Negative samples can include text and query statements that do not satisfy the query requirements from multiple texts retrieved using the query statement. Satisfying the query requirements can be understood as the credibility of the text retrieved by the query statement being greater than a credibility threshold. The credibility threshold can be an empirical value.

[0097] Figure 4 A schematic diagram illustrating the determination of a target strategy according to an embodiment of this disclosure is shown. Figure 5 The illustration shows a schematic diagram of a specific target strategy example according to an embodiment of the present disclosure.

[0098] For example, a target strategy matching the type of at least one problem can be determined based on a preset mapping relationship, according to the type of at least one problem. The preset mapping relationship characterizes the mapping relationship between problem types and target strategies, such as... Figure 4 As shown, the first preset mapping relationship 410 represents a mapping relationship between the first type of problem and the first target strategy. The second preset mapping relationship 420 represents a mapping relationship between the second type of problem and the second target strategy. And so on, the Nth preset mapping relationship 4N0 represents a mapping relationship between the Nth type of problem and the Nth target strategy. N is an integer greater than or equal to 1.

[0099] For example, question types can include: missing keywords, abnormal entity matching, abnormal word order, abnormal synonyms, etc. Corresponding target strategies can be pre-configured for each question type, such as... Figure 5 As shown, the mapping relationships are as follows: 541 for "Type 1 problem: missing keywords" and "Type 1 target strategy: highlighting the advantages of the target text compared to the text to be corrected with missing keywords"; 542 for "Type 2 problem: abnormal entity matching" and "Type 2 target strategy: highlighting the advantages of the target text compared to the text to be corrected with abnormal entity matching"; 543 for "Type 3 problem: abnormal word order" and "Type 3 target strategy: highlighting the advantages of the target text and the text with standard word order compared to the text to be corrected with abnormal word order"; and 544 for "Type 4 problem: abnormal synonyms" and "Type 4 target strategy: highlighting the advantages of the target text and the text with synonyms compared to the text to be corrected without abnormal synonyms". The distinguishing elements in each target strategy are keywords, entities, word order, and synonyms, respectively.

[0100] For example, to address the issue of keywords being located in different text fields, highlighting the advantages of the target text compared to the text to be corrected that lacks keywords could be done by stating that target text with keywords located in different text fields is superior to text without key information or that is irrelevant.

[0101] For example, to address the issue of keywords being located in different positions within a text field, highlight the advantages of the target text compared to the text to be corrected that lacks keywords. For instance, the target text with keywords located in different positions within the text field can be considered superior to text without key information or that is irrelevant.

[0102] For example, to address the issues of keywords being located in different text fields and keywords being located in different positions within text fields, the advantages of the target text compared to the text to be corrected that lacks keywords can be highlighted by combining the two aforementioned scenarios.

[0103] For example, for text with low confidence in the target query results, the low-confidence text can be repaired to obtain repaired positive samples. For the text of the positive samples, the keywords can be modified to obtain the target samples, thereby further increasing the base of sample amplification.

[0104] Because a target strategy is generated to produce training samples for specific types of problems that cause abnormal query results to be output by the retrieval model, and the target strategy indicates the difference between positive and negative samples for that specific type of problem, the retrieval model can learn how to reduce the output of abnormal query results based on these training samples.

[0105] Since multiple target query statements can be obtained by content enhancement based on the initial query statement, and multiple texts can be retrieved using a single target query statement, the target query results can include multiple initial texts.

[0106] Figure 6 A schematic diagram illustrating the generation of a target sample according to an embodiment of the present disclosure is shown.

[0107] like Figure 6 As shown, in this embodiment, based on the target strategy, modifying the target query statement and target query results using a large model to generate target samples may include the following operations: determining target text 602 from multiple initial texts 601; and modifying the target query statement 303 and target text 602 using a large model based on the target strategy to generate target samples.

[0108] In this embodiment of the disclosure, the matching degree between the target text 602 and the target query statement 303 is greater than or equal to a first predetermined threshold.

[0109] For example, the first predetermined threshold can be determined based on the matching degree between the text obtained that meets the query requirements and the query statement. Since the matching degree between the target text 602 and the target query statement 303 is greater than or equal to the first predetermined threshold, it can be determined that the target text 602 belongs to the positive sample text.

[0110] For example, the content of the target text 602 can be modified to have the same problems as those indicating missing keywords and / or abnormal entity matching and / or abnormal word order and / or text errors and / or abnormal keyword importance and / or abnormal synonyms, so that the target sample belongs to the text of the negative sample.

[0111] For example, the text to be corrected, which is about "discussing physical cooling methods for children or kids with fever", is modified using a large language model into the target text about "discussing drug treatment methods for adults with fever", and together with the target query statement, it is determined as the target sample.

[0112] Since the target text is the text that matches the target query, training samples generated based on modifications to the target text can construct a variety of negative samples.

[0113] In another embodiment of this disclosure, based on a target strategy, modifying the target query statement and target query results using a large model to generate target samples may further include the following operations: determining the text to be corrected from multiple initial texts; and using the large model to correct the text to be corrected to generate target text.

[0114] In this embodiment of the disclosure, the matching degree between the text to be corrected and the target query statement is less than a first predetermined threshold. Since the matching degree between the text to be corrected and the target query statement is less than the first predetermined threshold, it can be determined that the text to be corrected belongs to the negative sample text.

[0115] For example, text to be corrected that belongs to a negative sample can be restored to target text that belongs to a positive sample.

[0116] For example, the text to be corrected, which discusses "physical cooling methods for adults with fever", can be repaired using a large language model into the target text, which discusses "physical cooling methods for children or toddlers with fever".

[0117] Since the matching degree between the target query and the text to be corrected is less than the first predetermined threshold, the large model can be used to correct the text, which can improve the matching degree between the target query and the text, and turn the negative sample into a positive sample after modification. Therefore, the number of positive samples can be increased, which is beneficial to generating more target samples.

[0118] Figure 7 A schematic diagram illustrating the generation of target text according to an embodiment of the present disclosure is shown.

[0119] like Figure 7 As shown, in this embodiment, the large model is used to correct the text to be corrected and generate the target text. This may include the following operations: constructing prompt text 704 based on the target query statement 303, the text to be corrected 701, the matching degree 702 between the text to be corrected 701 and the target query statement 303, and the reference example 703. The prompt text 704 is input into the large model M, and the target text 602 is output.

[0120] In this embodiment of the disclosure, reference example 703 includes reference text that matches the reference query statement with a degree greater than a first predetermined threshold.

[0121] Figure 8 A schematic diagram illustrating a design based on an embodiment of this disclosure is shown.

[0122] For example, the 704 error message could be formatted as follows: Figure 8The template for prompt design 800 shown is generated. The template for prompt design 800 may include environment description 810, task definition 820, constraints 830, output format 840, and input data 850.

[0123] For example, environment description 810 could be: "You are a search engine optimization expert and need assistance in fixing matching issues in the retrieval model. These issues mainly manifest as inaccurate matching between the query statement and the query results, specifically including: reverse matching: high matching results have a lower matching degree than low matching results, and single query result anomalies: the similarity of the query results does not match the actual results."

[0124] Task definition 820 can be: "1. Analyze the problem: Carefully analyze the core intent and needs of users' query generalization problem, and determine the specific manifestations of similar abnormal query results. 2. Identify key features... 3. Clearly list the conditions required for positive samples...".

[0125] Constraint 830 can be "1. Authenticity: Conforms to users' real search habits. 2. Professionalism...".

[0126] Output format 840 allows you to specify the output format, etc.

[0127] Input data 850 can be used to provide explanations for the input data.

[0128] For example, the template of the prompt design 800 can be filled with the target query statement 303, the text to be corrected 701, the matching degree 702 between the text to be corrected 701 and the target query statement 303, and the reference example 703 to obtain the prompt text 704.

[0129] By constructing prompt text, large models can more clearly understand the task requirements and accurately output the target text that belongs to the positive sample.

[0130] According to another embodiment of this disclosure, generating a target sample by modifying the target query statement and target query results using a large model based on a target strategy may include the following operations: in response to determining that at least one question indicator keyword is missing, determining that the distinguishing element indicated by the target strategy is a keyword. Based on the target strategy, modifying the fields in the target text corresponding to the keywords in the target query statement using a large model to generate a first text. The target query statement, the target text, and the first text are then determined as the target sample.

[0131] For example, keywords may include words from the query statement.

[0132] For example, for target queries such as "Can physical cooling methods be used for children's fever?" and / or "Can physical cooling methods be used for children's fever?" and / or "Can wet towels be used to cool down children's fever?", the target text obtained regarding "physical cooling methods for children or children's fever" can be completely removed from the target text for the missing keyword "children and / or children and / or children" by changing the content to only discuss physical cooling methods for adults' fevers, without mentioning children and / or children and / or children. This ensures that no children-related words such as "children and / or children and / or children" appear in the text title, text summary, or text paragraphs, ensuring the professionalism of the content. At the same time, it ensures that it only involves physical cooling methods for adults' fevers, not simply replacing words, but adjusting the whole perspective to adults, resulting in the first text. The target strategy for addressing the missing keyword "fever" could be to completely remove all content related to "fever" and replace it with discussions of physical care methods for other childhood health issues (such as colds, coughs, etc.), without mentioning fever or body temperature. This ensures that no fever-related words such as "fever and / or elevated body temperature and / or high fever" appear in the text title, text summary, or text paragraphs. This maintains the content's focus on children, but by changing the health issue, rather than simply deleting words, the overall content direction is adjusted to obtain the first text.

[0133] Since the first text is the text of the negative sample obtained by replacing the target text in the positive sample, the first text constructed by replacing the keywords in the title, abstract, paragraph, etc. of the positive sample text with the large model can improve the model's ability to recognize keywords, thus addressing the issue of missing keywords.

[0134] According to another embodiment of this disclosure, generating a target sample by modifying the target query statement and target query results using a large model based on a target strategy may include the following operations: in response to determining that at least one question indicates an abnormal entity match, determining that the distinguishing element indicated by the target strategy is an entity. Based on the target strategy, modifying the fields in the target text corresponding to the entities in the target query statement using a large model to generate a second text. The target query statement, the target text, and the second text are then determined as the target sample.

[0135] For example, an entity match error could include entities that are distinctive or entities that are completely irrelevant. Entities could include topic entities, person entities, etc. A possible modification could be replacing the field corresponding to the entity with a completely dissimilar word.

[0136] For example, a target strategy for addressing entity matching anomalies could completely remove all content related to the entity "physical cooling" and instead discuss other treatment methods for children's fever (such as medication). This would avoid mentioning any content related to physical cooling, ensuring that no terms related to physical cooling, such as "physical cooling and / or warm water baths and / or ice packs," appear in the text title, text summary, or text paragraphs. The focus remains on children's fever, but only other treatment methods are discussed; the text is not simply replaced with words, but rather the overall direction of the treatment method is changed, resulting in a second text.

[0137] To address the issue of abnormal entity matching, a large model is used to replace multiple similar expressions of entities in the target text of positive samples to construct negative samples. Therefore, the model's ability to recognize similar expressions of entities can be improved.

[0138] According to another embodiment of this disclosure, modifying the target query statement and target query results using a large model based on a target strategy to generate a target sample may include the following operations: in response to determining that at least one question indicates an anomalous word order, determining that the distinguishing element of the target strategy indication is word order. Based on the target strategy, modifying the word order of content fields in the target text associated with the target query statement using a large model to generate third text. The target query statement, target text, and third text are then identified as the target sample.

[0139] For example, word order anomalies may include word order disorder, but with the same semantics as the non-disordered word order, or word order disorder, but with different semantics than the non-disordered word order.

[0140] Word order modification operations can include changing the positional order of words.

[0141] For example, modifying the word order of the content field "When a child's body temperature does not exceed 38.5℃, physical cooling can be given priority, as it is a safe and effective auxiliary method" in the target text associated with the target query statement to "When a child's body temperature does not exceed 38.5℃, auxiliary methods can be given priority, as it is a safe and effective physical cooling method" and / or "When a child's body temperature exceeds 38.5℃, physical cooling is not given priority, but it is a safe and effective auxiliary method" in the third text, etc.

[0142] To address the issue of abnormal word order, a large model is used to modify the word order of the target text in positive samples without altering the semantics, or to modify the word order of the target text in positive samples and then change the original semantics. This constructs difficult samples that are more challenging to identify, thereby improving the retrieval model's ability to recognize word order.

[0143] Since modifying the entire text content every time for each target text in the initial text or the target text obtained after repair would consume a lot of computing resources, this disclosure aims to minimize the amount of modification and save computing resources.

[0144] Figure 9 A schematic diagram illustrating the generation of a target sample according to another embodiment of this disclosure is shown.

[0145] like Figure 9 As shown, based on a target strategy for at least one problem, a large model is used to modify the target query statement and target query result associated with the initial query statement in the abnormal query result, generating a target sample. This may include the following operations: Based on the target strategy for at least one problem, determine the field to be modified 901. Based on the target strategy, use the large model to modify the target query statement and target query result for the field to be modified 901, generating an intermediate sample 902. In response to determining that the matching degree 904 between the intermediate sample 902 and the modification requirement 903 indicated by the target strategy is less than a second predetermined threshold, return to execute the determination operation for the field to be modified 901. In response to determining that the matching degree 904 between the intermediate sample 902 and the modification requirement 903 indicated by the target strategy is greater than or equal to the second predetermined threshold, determine the intermediate sample 902 as the target sample 603.

[0146] For example, the target strategy may also include modification requirement 903, which may be to make the modified text conform to semantic logic and to make the least amount of content modification to the text.

[0147] The field to be modified (901) can be determined based on the target strategy, defining the modification target and modification requirements. It refers to the necessary fields to be modified identified from the target query results based on the modification target and requirements.

[0148] The second predetermined threshold can be determined in advance based on the degree of matching between the text that meets the modification requirements and the modification requirements.

[0149] For example: While meeting the modification requirements, determine the minimum necessary modifications to the initial text in the target query results. Modification targets may include preserving all original paragraph structures unrelated to the modification goals, and / or preserving the initial sentence structure as much as possible, replacing only keywords or phrases, and / or retaining subject entities, place names, facility descriptions, etc., in the initial text if there are no conflicts.

[0150] Since the intermediate text is obtained by modifying the fields to be modified based on the target strategy, the target sample can be generated in an iterative manner with minimal modifications while still meeting the modification requirements, thereby reducing the consumption of computing resources.

[0151] In other embodiments of this disclosure, the sample generation method based on a large model may include, in addition to, the following: Figure 2 In addition to the operations S210~S220 shown, the operation may also include: based on at least one problem, using a large model to perform sample augmentation on the target sample to generate an amplified sample, so as to optimize the retrieval model using the target sample and the amplified sample.

[0152] For example, sample augmentation of a target sample can be performed by performing synonym replacement on the target text and target query statement in the target sample, or by adjusting the word order of the target text and target query statement in the target sample.

[0153] Figure 10 A schematic diagram illustrating the generation of training samples according to an embodiment of the present disclosure is shown.

[0154] like Figure 10 As shown, the initial query statement 301 corresponding to the abnormal query result 1001 is enhanced through synonymous expression, same-domain generalization, and cross-domain generalization to obtain the target query statement 303. The target query statement 303 is input into the retrieval model, and the target query result 1003 is output. The target query result 1003 may include multiple initial texts. From the multiple initial texts of the target query result 1003, the target text 602 with a matching degree greater than or equal to a first preset threshold with the target query statement is determined. The cause analysis of the abnormal query result 1001 can identify at least one problem, and the target strategy 1002 can be determined based on at least one problem. Based on the target strategy 1002, the target text 602 can be modified, and the target sample 603 can be obtained by combining it with the target query statement. The target sample 603 can be augmented to obtain the augmented sample 1004, and the augmented sample 1004 and the target sample 603 are used together as the training sample 1005. The training sample 1005 can be used to optimize the retrieval model and enhance its performance.

[0155] By augmenting all modified target samples, the number of training samples can be increased, which is beneficial for optimizing the retrieval model and improving its retrieval efficiency and accuracy.

[0156] For example, in generating augmented samples by using a large model to augment target samples based on at least one question, the operation may include: in response to determining that at least one question indicates an anomalous word order, adjusting the word order of the target samples using a large model to generate augmented samples.

[0157] In this embodiment of the disclosure, the amplified sample and the target sample express different word order but similar semantics.

[0158] For example, the word order of the text in the target sample can be adjusted so that the word order of the text in the augmented sample is different from that of the target sample, but the semantics are similar.

[0159] By performing synonym expansion based on word order adjustment on the target samples, sample augmentation is achieved, which can improve the retrieval model's ability to recognize word order changes.

[0160] For example, based on at least one problem, using a large model to augment target samples to generate expanded samples may include the operation of: in response to determining that at least one problem indicates a mismatch, using a large model to adjust the entities of the target samples to generate expanded samples.

[0161] In this embodiment of the disclosure, the amplified sample and the target sample express different content in different ways but have similar semantics; the matching anomaly includes at least one of the following: entity matching anomaly and keyword matching anomaly.

[0162] For example, entity matching anomalies can include abnormal entity match degree. Keyword matching anomalies can include missing keywords, etc.

[0163] Entity adjustment operations may include, for example, replacing words in the text of the target sample with synonyms, or changing the words indicating entities to different expressions.

[0164] By further adjusting the target samples based on entities and expanding them by changing the way synonyms or entity expressions are used, sample augmentation can be achieved, which can improve the retrieval model's ability to recognize word order changes.

[0165] Figure 11 A block diagram of a large-model-based sample generation apparatus according to an embodiment of the present disclosure is shown schematically.

[0166] like Figure 11 As shown, the sample generation device 1100 includes a receiving module 1110 and a generation module 1120.

[0167] The receiving module 1110 is used to receive at least one problem selected by the object on the problem analysis interface; the at least one problem is obtained by performing cause analysis on the abnormal query results output by the retrieval model.

[0168] The generation module 1120 is used to modify the target query statement and target query results using a large model based on at least one question to generate target samples, so as to optimize the retrieval model for at least one question based on the target samples.

[0169] According to embodiments of this disclosure, the target query statement is associated with the initial query statement in the abnormal query results; the target query results are obtained by using a retrieval model based on the target query statement.

[0170] According to embodiments of this disclosure, the generation module 1120 includes a first determining unit and a first modifying unit. The first determining unit is used to determine a target strategy based on at least one question. The target strategy indicates the distinguishing features between positive samples and negative text samples used to optimize the retrieval model. The first modifying unit is used to modify the target query statement and target query results using a large model based on the target strategy to generate target samples.

[0171] According to embodiments of this disclosure, the target query result includes: multiple initial texts. The first determining unit includes a first determining subunit and a first modifying subunit. The first determining subunit is used to determine the target text from the multiple initial texts. The matching degree between the target text and the target query statement is greater than or equal to a first predetermined threshold. The first modifying subunit is used to modify the target query statement and the target text based on a target strategy and using a large model to generate a target sample.

[0172] According to embodiments of this disclosure, the first determining unit further includes a second determining subunit and a second modifying subunit. The second determining subunit is used to determine the text to be modified from a plurality of initial texts. The matching degree between the text to be modified and the target query statement is less than a first predetermined threshold. The second modifying subunit is used to modify the text to be modified using a large model to generate the target text.

[0173] According to embodiments of this disclosure, a large model is used to correct the text to be corrected and generate target text, including: constructing prompt text based on a target query statement, the text to be corrected, the matching degree between the text to be corrected and the target query statement, and reference examples. Reference examples include reference texts whose matching degree with a reference query statement is greater than a first predetermined threshold. The prompt text is input into the large model, and the target text is output.

[0174] According to embodiments of this disclosure, the first determining unit includes a third determining subunit, a third modifying subunit, and a fourth determining subunit.

[0175] The third determination subunit is used to determine the distinguishing element of the target strategy indication as a keyword in response to the determination that at least one question indicator keyword is missing. The third modification subunit is used to modify the fields in the target text corresponding to the keywords in the target query statement based on the target strategy and using a large model to generate the first text. The fourth determination subunit is used to determine the target query statement, the target text, and the first text as the target sample.

[0176] According to embodiments of this disclosure, the first determining unit includes a fifth determining subunit, a fourth modifying subunit, and a sixth determining subunit.

[0177] The fifth determination subunit is used to determine the distinguishing element indicated by the target strategy as an entity in response to identifying at least one issue indicating an abnormal entity match. The fourth modification subunit is used to modify the fields in the target text corresponding to the entities in the target query statement based on the target strategy and using a large model to generate the second text. The sixth determination subunit is used to determine the target query statement, the target text, and the second text as the target sample.

[0178] According to embodiments of this disclosure, the first determining unit includes a seventh determining subunit, a fifth modifying subunit, and an eighth determining subunit. The seventh determining subunit is used to determine that the distinguishing element of the target strategy indication is word order in response to determining that at least one question indication word order is abnormal. The fifth modifying subunit is used to modify the word order of content fields in the target text associated with the target query statement using a large model based on the target strategy, generating third text. The eighth determining subunit is used to determine the target query statement, the target text, and the third text as the target sample.

[0179] According to embodiments of this disclosure, the generation module 1120 includes a second determining unit, a second modifying unit, a returning unit, and a third determining unit. The second determining unit is used to determine the field to be modified based on a target strategy for at least one problem. The second modifying unit is used to modify the target query statement and target query results for the field to be modified based on the target strategy using a large model, generating an intermediate sample. The returning unit is used to return to perform the determination operation for the field to be modified in response to determining that the matching degree between the intermediate sample and the modification requirement indicated by the target strategy is less than a second predetermined threshold. The third determining unit is used to determine the intermediate sample as a target sample in response to determining that the matching degree between the intermediate sample and the modification requirement indicated by the target strategy is greater than or equal to the second predetermined threshold.

[0180] According to embodiments of this disclosure, the sample generation apparatus 1100 further includes a content enhancement module. The content enhancement module is used to enhance the content of the initial query statement in the abnormal query results to generate a target query statement.

[0181] According to embodiments of this disclosure, the content enhancement module includes an element determination unit and a statement generation unit. The element determination unit is used to determine the query elements in the initial query statement. The statement generation unit is used to enhance the content of the initial query statement according to the query elements using a large model, and generate a target query statement.

[0182] According to embodiments of this disclosure, the query elements include a query scenario. Based on the query elements, a large model is used to enhance the content of the initial query statement to generate a target query statement, including: expanding the query scenario of the initial query statement using the large model to generate the target query statement.

[0183] According to embodiments of this disclosure, the query elements also include query roles. Based on the query elements, the initial query statement is enhanced using a large model to generate a target query statement, including: expanding the query roles of the initial query statement using the large model according to the query roles to generate the target query statement.

[0184] According to embodiments of this disclosure, the query elements also include query dimensions. Based on the query elements, the initial query statement is enhanced using a large model to generate a target query statement, including: based on the query dimensions, the query time of the initial query statement is expanded using the large model to generate the target query statement.

[0185] According to embodiments of this disclosure, the query elements also include a query domain. Based on the query elements, the initial query statement is enhanced using a large model to generate a target query statement, including: expanding the query domain of the initial query statement using a large model according to the query space to generate the target query statement.

[0186] According to embodiments of this disclosure, the sample generation apparatus 1100 further includes an amplification module. The amplification module is used to generate amplified samples by performing sample enhancement on target samples using a large model based on at least one problem, so as to optimize the retrieval model using the target samples and the amplified samples.

[0187] According to embodiments of this disclosure, the amplification module includes a first adjustment unit. The first adjustment unit is configured to, in response to determining at least one question indicating an abnormal word order, adjust the word order of the target sample using a large model to generate an amplified sample. The amplified sample and the target sample express content with different word order but similar semantics.

[0188] According to embodiments of this disclosure, the augmentation module includes a second adjustment unit. The second adjustment unit is configured to, in response to determining at least one question indicating a matching anomaly, adjust the target sample using a large model to generate an augmented sample. The augmented sample and the target sample express content in different ways but are semantically similar. The matching anomaly includes at least one of the following: entity matching anomaly and keyword matching anomaly.

[0189] According to embodiments of this disclosure, this disclosure also provides an intelligent agent for sample optimization, an electronic device, a readable storage medium, and a computer program product.

[0190] Figure 12 A block diagram of an intelligent agent for sample generation according to an embodiment of the present disclosure is shown schematically.

[0191] In embodiments of this disclosure, the von Neumann architecture in modern computer theory is inspired, such as... Figure 12As shown, the AI ​​agent 1200 may include three core modules: an input module 1210, an output module 1220, and a processing module 1230. The processing module 1230 may include a control unit 1231, a storage unit 1232, and a computing unit 1233.

[0192] The input module 1210 is responsible for receiving or sensing information such as queries, requests, instructions, signals, or data from the outside world (e.g., users or the external environment), and converting it into a format that the AI ​​agent 1200 can understand and process. The input module 1210 is the primary link for the AI ​​agent 1200 to interact with the outside world. It enables the AI ​​agent 1200 to efficiently and accurately obtain necessary "sensory" information from the outside world and respond to this information.

[0193] In the example, the input information received by the input module 1210 can be at least one optimization problem described above.

[0194] In the example, the processing module 1230 is the core support for the AI ​​agent 1200's ability to handle complex tasks. The processing module 1230 can determine the target task based on the input information received by the input module 1210, determine the large model based on the target task, and execute the sample generation method based on the large model described above by calling the large model to output the target sample.

[0195] In the example, the control unit 1231 in the processing module 1230 will continuously interact with the storage unit 1232, the arithmetic unit 1233, and / or the output module 1220 during operation. However, it should be noted that in the embodiments of this disclosure, the control unit 1231 initiates communication with the storage unit 1232, the arithmetic unit 1233, and / or the output module 1220 as a single initiator, and there is no communication coupling between the storage unit 1232, the arithmetic unit 1233, and the output module 1220.

[0196] In the example, the performance of the control unit 1231 is closely related to the large model on which the AI ​​agent 1200 is based. To fully leverage the capabilities of the large language model, the internal structure of the control unit 1231 can be designed to be highly configurable and scalable to handle various types of tasks and requirements in real-world scenarios.

[0197] Storage unit 1232 can be responsible for remembering information such as historical dialogues and event streams. Configuration information, target text, and data resources generated in each round can be included in storage unit 1232.

[0198] In the example, after receiving a configuration generation request, the AI ​​agent 1200 can determine the configuration intent from the initial text using an intent recognition model. The configuration intent can be stored in the storage unit 1232. The AI ​​agent 1200 can retrieve relevant data resources from the storage unit 1232 and feed them back to the control unit 1231. Then, the control unit 1231 can use the returned data resources to obtain the configuration data corresponding to the initial text. It can also retrieve relevant text data from the storage unit 1232 and feed it back to the control unit 1231. Then, the control unit 1231 can use the returned text data to obtain the target text and pass the target text and configuration data to the output module 1220.

[0199] The operation unit 1233 can be viewed as a predefined tool library. Renderers and display controls, as mentioned earlier, can be included in the operation unit 1233.

[0200] In the example, when the AI ​​agent 1200 needs to render multiple output data, it can call the relevant renderer and display controls from the computing unit 1233 and feed them back to the control unit 1231. Then, the control unit 1231 can use the fed-back renderer and display controls to render the first search result and pass it to the output module 1220. It is understandable that although large language models have excellent language understanding and generation capabilities, like humans, the tasks they can solve without any tools are very limited. When the AI ​​agent 1200 is given the ability to call tools, it can perform tasks such as using a calculator to complete mathematical calculations, using Python to complete data analysis, and using a search engine to complete prediction tasks.

[0201] In the example, output module 1220 can output the target sample described above.

[0202] The AI ​​agent 1200 according to the embodiments of this disclosure can simply and effectively improve the level of intelligence, and enhance flexibility and versatility.

[0203] According to an embodiment of the present disclosure, an electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method described above.

[0204] According to embodiments of the present disclosure, a non-transitory computer-readable storage medium stores computer instructions, wherein the computer instructions are used to cause a computer to perform the methods described above.

[0205] According to an embodiment of this disclosure, a computer program product includes a computer program that, when executed by a processor, implements the method described above.

[0206] Figure 13 A block diagram schematically illustrates an electronic device suitable for implementing a large-model-based sample generation method according to embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0207] like Figure 13 As shown, device 1300 includes a computing unit 1301, which can perform various appropriate actions and processes according to a computer program stored in read-only memory (ROM) 1302 or a computer program loaded from storage unit 1308 into random access memory (RAM) 1303. The RAM 1303 may also store various programs and data required for the operation of device 1300. The computing unit 1301, ROM 1302, and RAM 1303 are interconnected via bus 1304. Input / output (I / O) interface 1305 is also connected to bus 1304.

[0208] Multiple components in device 1300 are connected to I / O interface 1305, including: input unit 1306, such as keyboard, mouse, etc.; output unit 1307, such as various types of monitors, speakers, etc.; storage unit 1308, such as disk, optical disk, etc.; and communication unit 1309, such as network card, modem, wireless transceiver, etc. Communication unit 1309 allows device 1300 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0209] The computing unit 1301 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1301 performs the various methods and processes described above, such as the large model-based sample optimization method. For example, in some embodiments, the large model-based sample optimization method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1308. In some embodiments, part or all of the computer program may be loaded and / or installed on device 1300 via ROM 1302 and / or communication unit 1309. When the computer program is loaded into RAM 1303 and executed by the computing unit 1301, one or more steps of the large model-based sample optimization method described above may be performed. Alternatively, in other embodiments, computing unit 1301 may be configured by any other suitable means (e.g., by means of firmware) to perform a sample optimization method based on a large model.

[0210] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0211] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0212] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0213] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0214] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0215] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, distributed system servers, or servers incorporating blockchain technology.

[0216] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0217] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A sample generation method based on a large model, comprising: The receiving object selects at least one problem on the problem analysis interface; The at least one problem is obtained by analyzing the causes of abnormal query results output by the retrieval model; as well as Based on the at least one problem, a target strategy is determined; wherein the target strategy indicates the distinguishing factors between positive and negative samples for optimizing the retrieval model; The initial query statement in the abnormal query results is augmented to obtain the target query statement; Based on the target strategy, the target query statement and target query results are modified using the large model to generate target samples, so as to optimize the retrieval model for the at least one question based on the target samples; The target query statement is associated with the initial query statement in the abnormal query results; the target query results are obtained by using the retrieval model based on the target query statement.

2. The method according to claim 1, wherein, The target query results include: multiple initial texts; The step of modifying the target query statement and the target query results using the large model based on the target strategy to generate the target sample includes: Determine target text from the plurality of initial texts; wherein the matching degree between the target text and the target query statement is greater than or equal to a first predetermined threshold; and Based on the target strategy, the target query statement and the target text are modified using the large model to generate the target sample.

3. The method according to claim 2, wherein, The step of modifying the target query statement and the target query results using the large model based on the target strategy to generate the target sample further includes: The text to be corrected is determined from the plurality of initial texts; wherein the degree of matching between the text to be corrected and the target query statement is less than the first predetermined threshold. The large model is used to correct the text to be corrected, and the target text is generated.

4. The method according to claim 3, wherein, The step of using the large model to correct the text to be corrected and generating the target text includes: Based on the target query, the text to be corrected, and the matching degree between the text to be corrected and the target query, and reference examples, a prompt text is constructed; wherein, the reference examples include reference texts whose matching degree with the reference query is greater than the first predetermined threshold; and The prompt text is input into the large model, and the target text is output.

5. The method according to claim 4, wherein, The process of modifying the target query statement and the target text using the large model based on the target strategy to generate the target sample includes: In response to determining that at least one question indicator keyword is missing, the distinguishing element of the target strategy indicator is determined to be a keyword; Based on the target strategy, the large model is used to modify the fields in the target text that correspond to the keywords in the target query statement, generating the first text; and The target query statement, the target text, and the first text are determined as the target sample.

6. The method according to claim 5, wherein, The process of modifying the target query statement and the target text using the large model based on the target strategy to generate the target sample includes: In response to determining that the at least one problem indicates an abnormal entity matching degree, the distinguishing element indicated by the target policy is determined to be an entity; Based on the target strategy, the large model is used to modify the fields in the target text that correspond to the entities in the target query statement, generating a second text; and The target query statement, the target text, and the second text are determined as the target sample.

7. The method according to claim 6, wherein, The process of modifying the target query statement and the target text using the large model based on the target strategy to generate the target sample includes: In response to determining that the at least one problem indicates an abnormal word order, the distinguishing element indicated by the target strategy is determined to be word order; Based on the target strategy, the word order of content fields associated with the target query statement in the target text is modified using the large model to generate third text; and The target query statement, the target text, and the third text are determined as the target sample.

8. The method according to any one of claims 1-7, wherein, The process of modifying the target query statement and target query results using the large model based on the target strategy to generate the target sample includes: Based on the target strategy for the at least one of the aforementioned issues, determine the fields to be modified; Based on the target strategy, the target query statement and target query results are modified using a large model to modify the fields to be modified, thereby generating intermediate samples; In response to determining that the matching degree between the intermediate sample and the modification requirement indicated by the target strategy is less than a second predetermined threshold, the process returns to perform the determination operation for the field to be modified. In response to determining that the matching degree between the intermediate sample and the modification requirement indicated by the target strategy is greater than or equal to the second predetermined threshold, the intermediate sample is determined to be the target sample.

9. The method according to claim 1, wherein, The step of enhancing the content of the initial query statement in the abnormal query results to generate the target query statement includes: Determine the query elements in the initial query statement; and Based on the query elements, the initial query statement is enhanced using the large model to generate the target query statement.

10. The method according to claim 9, wherein, The query elements include the query scenario; The step of enhancing the content of the initial query statement using the large model based on the query elements to generate the target query statement includes: Based on the query scenario, the initial query statement is expanded using the large model to generate the target query statement.

11. The method according to claim 10, wherein, The query elements also include: query role; The step of enhancing the content of the initial query statement using the large model based on the query elements to generate the target query statement includes: Based on the query roles, the query roles of the initial query statement are expanded using the large model to generate the target query statement.

12. The method according to claim 11, wherein, The query elements also include: query dimensions; The step of enhancing the content of the initial query statement using the large model based on the query elements to generate the target query statement includes: Based on the stated query dimensions, the initial query statement is expanded using the large model to generate the target query statement.

13. The method according to claim 12, wherein, The query elements also include: the query domain; The step of enhancing the content of the initial query statement using the large model based on the query elements to generate the target query statement includes: Based on the query domain, the query domain of the initial query statement is expanded using the large model to generate the target query statement.

14. The method according to claim 13, further comprising: Based on the at least one of the aforementioned problems, the target sample is augmented using a large model to generate amplified samples, so as to optimize the retrieval model using the target sample and the amplified samples.

15. The method according to claim 14, wherein, The step of using a large model to augment the target sample and generate expanded samples based on the at least one problem includes: In response to determining that the at least one question indicates an abnormal word order, the word order of the target sample is adjusted using the large model to generate the augmented sample; wherein the word order of the content expressed by the augmented sample is different from that of the target sample but the semantics are similar.

16. The method according to claim 14 or 15, wherein, The step of using a large model to augment the target sample and generate expanded samples based on the at least one problem includes: In response to determining that the at least one question indicates a matching anomaly, entity adjustment is performed on the target sample using the large model to generate the augmented sample; wherein the augmented sample expresses content differently from the target sample but is semantically similar; the matching anomaly includes at least one of the following: entity matching anomaly and keyword matching anomaly.

17. A sample generation device based on a large model, comprising: The receiving module is used to receive at least one question selected by the object on the question analysis interface; The at least one problem is obtained by analyzing the causes of abnormal query results output by the retrieval model; as well as A generation module is configured to determine a target strategy based on the at least one question; wherein the target strategy indicates the distinguishing factors between positive and negative samples for optimizing the retrieval model; The content enhancement module is used to enhance the content of the initial query statement in the abnormal query results to obtain the target query statement; The generation module is used to modify the target query statement and target query results based on the target strategy using the large model to generate target samples, so as to optimize the retrieval model based on the target samples for the at least one question; The target query statement is associated with the initial query statement in the abnormal query results; the target query results are obtained by using the retrieval model based on the target query statement.

18. An intelligent agent for sample generation, comprising: The input module is used to receive at least one question selected by the object on the question analysis interface; A processing module is configured to determine a target task based on the at least one question received by the input module, determine a target large model based on the target task, and execute the method described in any one of claims 1-16 by calling the target large model to obtain target samples for optimizing the retrieval model; as well as An output module is used to output the target sample obtained by the processing module.

19. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-16.

20. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-16.

21. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-16.

Images