Data generation method and device based on large model and product

Abstract

The invention provides a data generation method and device based on a large model, electronic equipment, a storage medium and a computer program product, relates to the technical field of artificial intelligence, in particular to the technical fields of large models, natural language understanding, retrieval enhancement generation and the like, and can be applied to intelligent question answering and code generation scenes. According to the specific implementation scheme, standard terms corresponding to keywords in a query request are determined from mapping relation data, and the mapping relation data represent the mapping relation between the keywords and the standard terms; adjusting the query request according to the standard terms, and generating an adjusted query request; performing data retrieval according to the adjusted query request to obtain a retrieval result; and adopting an artificial intelligence large model to generate reply data aiming at the query request according to the retrieval result. According to the method, the expression accuracy and comprehensiveness of the adjusted query request are improved, and the accuracy of the reply data obtained by processing the query request based on the artificial intelligence large model is improved.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, specifically to the fields of large-scale artificial intelligence models, natural language understanding, and retrieval enhancement generation. In particular, it relates to a data generation method, device, electronic device, storage medium, and computer program product based on a large-scale model, which can be applied to intelligent question answering and code generation scenarios. Background Technology

[0002] In codebase knowledge retrieval applications, existing RAG (Retrieval Augmented Generation) methods are widely used. By using retrieval results from private domain knowledge such as codebases / knowledge bases / web searches as context, the answers generated by large language models become more accurate. Users tend to express their query needs using natural language, and these expressions are often vague, incomplete, or ambiguous, making it difficult for traditional RAG processes to accurately capture the user's true intent. Summary of the Invention

[0003] This disclosure provides a method, apparatus, electronic device, storage medium, and computer program product for generating data based on a large model.

[0004] According to the first aspect, a data generation method based on a large model is provided, including: determining the standard terms corresponding to the keywords in the query request from mapping relationship data, wherein the mapping relationship data represents the mapping relationship between keywords and standard terms; adjusting the query request according to the standard terms to generate an adjusted query request; performing data retrieval based on the adjusted query request to obtain retrieval results; and using an artificial intelligence large model to generate response data for the query request based on the retrieval results.

[0005] According to the second aspect, a data generation apparatus based on a large model is provided, comprising: a term mapping unit configured to determine standard terms corresponding to keywords in a query request from mapping relationship data, wherein the mapping relationship data represents the mapping relationship between keywords and standard terms; a request adjustment unit configured to adjust the query request according to the standard terms to generate an adjusted query request; a data retrieval unit configured to perform data retrieval according to the adjusted query request to obtain retrieval results; and a data generation unit configured to use an artificial intelligence large model to generate response data for the query request based on the retrieval results.

[0006] According to a third aspect, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform a method as described in any implementation of the first aspect.

[0007] According to a fourth aspect, a non-transitory computer-readable storage medium is provided that stores computer instructions for causing a computer to perform the method described in any implementation of the first aspect.

[0008] According to a fifth aspect, a computer program product is provided, comprising: a computer program that, when executed by a processor, implements the method as described in any implementation of the first aspect.

[0009] According to the technology disclosed herein, a data generation method and apparatus based on a large model are provided. The method involves determining the standard terms corresponding to keywords in a query request from mapping relationship data, where the mapping relationship data represents the mapping relationship between keywords and standard terms; adjusting the query request based on the standard terms to generate an adjusted query request; performing data retrieval based on the adjusted query request to obtain retrieval results; and employing an artificial intelligence large model to generate response data for the query request based on the retrieval results. This method improves the accuracy and comprehensiveness of the adjusted query request by determining the standard terms for keywords in the query request through mapping relationship data, which helps the artificial intelligence large model understand the query intent of the target object and improves the accuracy of the response data obtained from processing the query request based on the artificial intelligence large model.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0011] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein: Figure 1 This is an exemplary system architecture diagram that can be applied to an embodiment of this disclosure; Figure 2 This is a flowchart of an embodiment of the data generation method based on a large model according to the present disclosure; Figure 3 This is a schematic diagram illustrating an application scenario of the data generation method based on a large model according to this embodiment; Figure 4 This is a flowchart of yet another embodiment of the data generation method based on a large model according to the present disclosure; Figure 5 This is a flowchart of an embodiment of the method for generating and updating mapping relationship data according to this disclosure; Figure 6 This is a structural diagram of an embodiment of the data generation apparatus based on a large model according to the present disclosure; Figure 7 This is a schematic diagram of the structure of a computer system suitable for implementing the embodiments of the present disclosure. Detailed Implementation

[0012] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0013] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0014] Figure 1 An exemplary architecture 100 is shown that can be applied to the large model-based data generation method and apparatus disclosed herein.

[0015] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. The communication connections between terminal devices 101, 102, and 103 form a network topology. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0016] Terminal devices 101, 102, and 103 can be hardware or software that supports network connectivity for data interaction and processing. When terminal devices 101, 102, and 103 are hardware, they can be various electronic devices that support network connectivity, information acquisition, interaction, display, and processing functions, including but not limited to smartphones, tablets, e-book readers, laptops, and desktop computers. When terminal devices 101, 102, and 103 are software, they can be installed in the aforementioned electronic devices. They can be implemented as, for example, multiple software programs or software modules to provide distributed services, or as a single software program or software module. No specific limitations are imposed here.

[0017] Server 105 can be a server that provides various services, such as a server that receives query requests from target objects through terminal devices 101, 102, and 103, adjusts the query requests based on mapping relationship data representing the mapping relationship between keywords and standard terms, and generates response data based on the search results corresponding to the adjusted query requests using a large artificial intelligence model. As an example, server 105 could be a cloud server.

[0018] It should be noted that a server can be either hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When the server is software, it can be implemented as multiple software programs or software modules (such as software programs or software modules used to provide distributed services), or as a single software program or software module. No specific limitations are made here.

[0019] It should also be noted that the large-model-based data generation method provided in the embodiments of this disclosure is generally executed by a server, but the possibility of it being executed by a terminal device, or by a combination of the server and the terminal device, is not excluded. Accordingly, the various parts (e.g., various units) of the large-model-based data generation apparatus can be all located in the server, all located in the terminal device, or separately located in the server and the terminal device.

[0020] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included. When the electronic devices running the large-model-based data generation method do not require data transmission with other electronic devices, the system architecture may consist only of the electronic devices (e.g., terminal devices or servers) running the large-model-based data generation method.

[0021] Please refer to Figure 2 , Figure 2 A flowchart illustrating a data generation method based on a large model, provided for embodiments of this disclosure. Flowchart 200 includes the following steps: Step 201: Determine the standard terms corresponding to the keywords in the query request from the mapping relationship data.

[0022] In this embodiment, the execution entity of the data generation method based on the large model (e.g., Figure 1The server (in the system) can retrieve mapping relationship data remotely or locally via wired or wireless network connections, and determine the standard terms corresponding to the keywords in the query request from the mapping relationship data. The mapping relationship data represents the mapping relationship between keywords and standard terms. The mapping relationship between keywords and standard terms can be one-to-many or one-to-one. Taking a one-to-many mapping relationship as an example, the same keyword has mapping relationships with multiple standard terms.

[0023] Query requests can take at least one data modality, including text, voice, image, and video. Standard terminology refers to professional vocabulary or expressions with clear connotations and denotations that have been defined and standardized by authoritative institutions, professional organizations, or industry consensus within a specific field (such as disciplines, industries, professional systems, etc.).

[0024] In different application areas or scenarios, mapping relationships can be set specifically. For example, in the code domain, standard terms are mostly syntax, function names, concepts, etc., defined by programming language specifications. Standard terms can be expressed in Chinese or in code, while keywords often appear as colloquial expressions, aliases, or simplified names used by developers. For example, the standard term "Object Oriented Programming (OOP)" corresponds to keywords such as "object programming," "OO programming," and "object-based development model"; "Exception Handling" corresponds to keywords such as "error handling," "error catching," "exception catching mechanism," and "try-catch usage"; and "Hash Table" corresponds to keywords such as "hash table," "hash mapping," "key-value pair storage structure," and "dict (Python colloquialism)."

[0025] In the medical field, standard terminology is mostly medically standardized naming (such as disease names corresponding to ICD (International Classification of Diseases) codes, anatomical terms, etc.), while keywords are colloquial descriptions used by patients or non-professionals. For example, keywords for "acute bronchitis" include "cough with phlegm," "tracheal inflammation," and "persistent cough caused by a cold"; keywords for "hypertension" include "high blood pressure," "dizziness with elevated blood pressure," and "high systolic blood pressure"; and keywords for "type 2 diabetes mellitus" include "adult-onset diabetes," "non-insulin-dependent diabetes," and "high blood sugar."

[0026] In the legal field, standard terminology refers to concepts explicitly stated in legal provisions, while keywords are often simplified or everyday expressions used by the general public to describe legal phenomena. Keywords for "contract dispute" include "disputes over agreements," "contract not being fulfilled," and "a signed contract is invalid." Keywords for "justifiable defense" include "self-defense," "retaliating after being attacked," and "injuring others to protect oneself." Keywords for "joint liability" include "sharing responsibility," "joint compensation," and "the guilty party may run away, but the guilty party can't escape responsibility (in scenarios involving related liability)."

[0027] Upon receiving a query request, the keywords in the query request are identified. For each keyword, the corresponding standard terminology is determined from the mapping relationship data. For example, in the code domain, the query request is used to query code, and the standard terms corresponding to the keywords in the query request are generally code terms.

[0028] Step 202: Adjust the query request according to standard terminology and generate the adjusted query request.

[0029] In this embodiment, the aforementioned execution entity can adjust the query request according to standard terminology to generate an adjusted query request.

[0030] As an example, for each keyword in the query request, the keyword is replaced with the standard term corresponding to that keyword to generate an adjusted query request.

[0031] As another example, for each keyword in the query request, the standard term corresponding to that keyword is combined with the keyword to provide supplementary explanation, thereby generating an adjusted query request.

[0032] In some optional implementations of this embodiment, the execution entity can perform step 202 as follows: The first step is to use a pre-trained intent recognition model to determine the query intent of the query request based on standard terminology.

[0033] Intent recognition models are used to represent the correspondence between query requests and query intents. Examples include neural network models such as BERT (Bidirectional Encoder Representations from Transformers), LSTM (Long Short-Term Memory), and NN (Recurrent Neural Networks). The intent recognition model is input into the query request and the standard terms corresponding to the keywords in the query request; the model then outputs the query intent.

[0034] For example, an AI big data model can be used as an intent recognition model. The query request, the standard terms corresponding to the keywords in the query request, and the preset intent recognition prompts (e.g., please deeply identify the query intent of the query request) are input into the AI ​​big data model, and the AI ​​big data model outputs the query intent.

[0035] The second step involves using a large-scale artificial intelligence model to reconstruct the query request based on the query intent and generate an adjusted query request.

[0036] In this implementation, the query intent and query request are input into the AI ​​big data model. The AI ​​big data model expands and / or rewrites the query request to generate an adjusted query request with accurate and comprehensive description.

[0037] This implementation provides a specific method for generating the adjusted query request, and by combining the intent recognition model and the large-scale artificial intelligence model, it improves the accuracy and comprehensiveness of the adjusted query request.

[0038] In some optional implementations of this embodiment, the execution entity can perform the first step described above to determine the query intent of the query request in the following manner: First, replace the keywords in the query request with standard terms to obtain the initial adjusted request.

[0039] For each keyword in the query request, replace that keyword in the query request with the standard term corresponding to that keyword to obtain the initial adjusted request.

[0040] When a keyword has multiple corresponding standard terms, the query request and the multiple standard terms corresponding to the keyword in the query request can be input into the intent recognition model. The intent recognition model will then determine the target term from the multiple standard terms to replace the keyword in the query request.

[0041] Then, by combining the association characteristics of the query request, the query intent of the initially adjusted request is determined.

[0042] The associated characteristics of a query request include at least one of the following dimensions: a composite query intent dimension indicating whether the query request has multiple associated query intents; a request context dimension indicating whether the query request depends on historical query requests; a knowledge domain dimension (the scope of the target knowledge set explicitly specified by the user); and a database dimension (associated database identifiers and version information). Taking intelligent question-answering generation and code retrieval scenarios as examples, the database dimension represents the associated code repository identifiers and version information.

[0043] The intent recognition model uses the associated features of the query request and the initial adjusted request as the basis for data processing to generate the query intent of the initial adjusted request.

[0044] In this implementation, the query request is first replaced with standard terms, and then the query intent of the initial adjusted request after term replacement is determined by combining the association characteristics of the query request, thereby improving the accuracy of the query intent.

[0045] Step 203: Perform data retrieval based on the adjusted query request to obtain the retrieval results.

[0046] In this embodiment, the aforementioned execution entity can perform data retrieval based on the adjusted query request and obtain the retrieval results.

[0047] As an example, firstly, the adjusted query request is subject-specifically identified to determine its general category, and pre-defined category tags (such as "tourism," "repair," and "restaurant") are associated with the subject. Then, these category tags are used to filter the knowledge base for documents belonging to the corresponding subject, quickly narrowing the search scope and avoiding interference from irrelevant information. Next, core keywords are extracted from the adjusted query request and expanded with synonyms and near-synonyms. Then, the expanded keywords are used for precise matching within the filtered document set, and the documents are sorted according to their matching degree. Finally, the top few most relevant documents are output as search results, with each document's content segments matching the keywords annotated.

[0048] As another example, firstly, the adjusted query request is analyzed in context to understand its underlying intent, and then converted into a vector representation containing semantic information. Next, the semantic similarity between each document and the query vector is calculated in the knowledge base, initially filtering out documents with high semantic similarity. Then, user profile data (such as historical query records, preference tags, etc., with general preferences used for new users) is invoked to analyze the user's personalized needs (e.g., if the user has historically focused on places with "high cost-performance ratio," then documents of this type are prioritized in the search). Finally, the initially filtered documents are re-ranked based on semantic similarity and user preferences, prioritizing the display of documents that both match the query semantics and the user's preferences.

[0049] Databases or knowledge bases used for data retrieval can be private knowledge bases belonging to individuals or companies, or public internet knowledge bases.

[0050] In some optional implementations of this embodiment, the execution entity can perform step 203 as follows: The first step is to determine the retrieval task based on the adjusted query request. The retrieval task includes the detection target, retrieval type, search terms, and retrieval strategy.

[0051] As an example, the adjusted query request is first decomposed in a structured manner to extract the core object and limiting conditions, clarify the retrieval target, and obtain information that contains both the core object and all limiting conditions to ensure that the results are both centered on the core topic and meet the specific constraints.

[0052] Then, the search type is determined based on the nature of the core object and the limiting conditions: if the core object is a clear category of things, a "topic-focused search" is used; if the limiting conditions involve attributes, scope, etc., a "condition-filtering search" is added.

[0053] Next, basic search terms are extracted from the core objects, constraint search terms are extracted from the limiting conditions, and the coverage of search terms is enhanced by expanding with synonyms (such as including words with similar expressions in the search term library) to avoid missed detections due to differences in expression.

[0054] Finally, a search strategy is developed. First, a preliminary search is performed using basic search terms to obtain a set of information covering the core objects. Then, the set is filtered layer by layer using constraint search terms to remove content that does not meet the constraints. Finally, cross-validation (checking whether the information simultaneously meets all constraints) is used to ensure the accuracy of the results, forming the final search task.

[0055] As another example, the adjusted query request and the preset retrieval task generate prompts (for example, in the RAG scenario, the detection target, retrieval type, search terms and retrieval strategy in the retrieval task are determined based on the adjusted query request) are input into the big AI model, and the big AI model determines the retrieval task.

[0056] The second step is to execute the search task and obtain the search results.

[0057] The retrieval task was executed strictly in accordance with the detection target, retrieval type, search terms, and retrieval strategy to obtain the retrieval results.

[0058] In this implementation, the query request is first adjusted to determine the retrieval task, and then the retrieval task is executed to determine the retrieval results, which improves the accuracy of the retrieval results and their adaptability to the query request.

[0059] Step 204: Using a large-scale artificial intelligence model, generate response data for the query request based on the search results.

[0060] In this implementation, a large artificial intelligence model is used to generate response data for the query request based on the search results.

[0061] As an example, the search results and the adjusted query request can be input into the AI ​​big data model. Based on its powerful data processing and logical analysis capabilities, the AI ​​big data model generates response data for the query request.

[0062] As another example, in order to further improve the accuracy of the response data and its suitability to the target object of the query request, the search results and the adjusted query request are input into the AI ​​big data model. The AI ​​big data model calls the target object's profile data (e.g., preferred expression style, text format) to generate response data adapted to the target object.

[0063] Large-scale artificial intelligence models (or simply large models) refer to a class of artificial intelligence models with a large number of parameters built from artificial neural networks, such as large language models, large vision models, multimodal large models, and basic science large models. Taking a large language model as an example, it is a large-scale language model built based on deep learning technology, primarily used for natural language processing tasks. Through training on large-scale data, it learns language patterns and structures, enabling it to generate natural language text or understand natural language input. This embodiment may specifically employ a multimodal large language model, which typically includes the following modules: Input module: Receives multimodal data such as text, images, videos, and input information from users, as well as query requests such as questions, instructions, or dialogue content.

[0064] Preprocessing module: preprocesses the input multimodal data. For example, text data preprocessing includes word segmentation, stop word removal, text cleaning, etc., to convert the text into a form that the model can process.

[0065] The encoding module encodes the preprocessed multimodal data into vector form so that the model can understand and process it. Common encoding methods include word embeddings and encoders in the Transformer architecture. Word embeddings include, for example, Word2Vec (Words to Vector) and GloVe (Global Vectors for Word Representation).

[0066] Model module: The core component, typically based on a deep learning architecture (such as Transformer), responsible for processing encoded data vectors and performing language understanding and generation. The model learns complex patterns and semantic relationships of language through multi-layered neural network structures.

[0067] Decoding module: Decodes the model's output vectors into natural language text, images, videos, and input information, generating response data to user input. Decoding methods can include greedy decoding, beam search, etc.

[0068] Output module: Outputs the decoded text in a user-readable format, such as text, images, and videos displayed on the screen.

[0069] See also Figure 3 , Figure 3 This is a schematic diagram 300 illustrating an application scenario of the large-model-based data generation method according to this embodiment. Target user 301 sends a query request to server 303 via terminal device 302. Upon receiving the query request, the server first determines the standard terms corresponding to the keywords in the query request from the mapping relationship data, where the mapping relationship data represents the mapping relationship between keywords and standard terms. Then, it adjusts the query request based on the standard terms to generate an adjusted query request. Next, it performs data retrieval based on the adjusted query request to obtain the retrieval results. Finally, using an artificial intelligence large-model, it generates response data for the query request based on the retrieval results and displays the response data to the target user via the terminal device.

[0070] This embodiment provides a data generation method based on a large model. It determines the standard terms corresponding to keywords in a query request from mapping relationship data, where the mapping relationship data represents the mapping relationship between keywords and standard terms. The query request is adjusted based on the standard terms to generate an adjusted query request. Data retrieval is performed based on the adjusted query request to obtain retrieval results. An artificial intelligence large model is used to generate response data for the query request based on the retrieval results. This method, by determining the standard terms for keywords in the query request through mapping relationship data to adjust the query request, improves the accuracy and comprehensiveness of the adjusted query request, helps the artificial intelligence large model understand the query intent of the target object, and improves the accuracy of the response data obtained by processing the query request based on the artificial intelligence large model.

[0071] In some optional implementations of this embodiment, the execution entity can generate the mapping relationship data in the following manner: First, for the terminology description data and non-terminology description data in the data set, based on the co-occurrence characteristics between standard terms in the terminology description data and keywords in the non-terminology description data, sub-mapping relationship data representing the mapping relationship between standard terms and keywords in the data set is determined.

[0072] Terminological data and non-terminal data in the same data set are two related types of data. For example, in the medical field, the terminological data in a data set might be doctors' diagnostic cases, while the non-terminal data might be patients' symptom data described in natural language.

[0073] Co-occurrence features characterize the co-occurrence relationship between standard terms and keywords in the data set, such as co-occurrence frequency. For standard terms in the terminology description data, the co-occurrence features between them and each keyword in the non-terminology description data can be determined, thereby establishing a mapping relationship between standard terms and keywords with relatively close co-occurrence relationships (e.g., co-occurrence frequency greater than a preset frequency threshold).

[0074] For standard terms and keywords that have mapping relationships in the same data set, they are combined into sub-mapping relationship data, thus obtaining multiple sub-mapping relationship data corresponding one-to-one with multiple data sets in the data set.

[0075] Then, by combining the sub-mapping relationship data of each data group in the data set, mapping relationship data is generated.

[0076] For each sub-mapping relationship data of a data group in the data set, keywords that have a mapping relationship with the same standard term are combined together to generate mapping relationship data.

[0077] This implementation provides a specific method for generating mapping relationship data, which improves the accuracy and comprehensiveness of the mapping relationship data.

[0078] In some optional implementations of this embodiment, the execution entity can update the mapping relationship data in the following manner: The first step, in response to the data update condition being met, is to determine the sub-mapping relationship data corresponding to the updated data group based on the co-occurrence characteristics between standard terms in the terminology description data and keywords in the non-terminology description data, for the updated data group in the data group set.

[0079] Data update conditions can be flexibly set according to actual conditions. For example, the number of updated data groups in the data set may exceed a preset threshold, or the interval between the last update and the current update time may exceed a preset time threshold. The updated data group can be a data group obtained by updating an existing data group, or a newly added data group in the data set.

[0080] For standard terms in the terminology description data of the updated data group, the co-occurrence characteristics between them and each keyword in the non-terminology description data of the same group can be determined, thereby establishing a mapping relationship between standard terms and keywords with relatively close co-occurrence relationships (e.g., co-occurrence frequency greater than a preset frequency threshold).

[0081] The second step is to combine the updated sub-mapping relationship data corresponding to the data group with the unupdated sub-mapping relationship data corresponding to the data group in the data group set to generate mapping relationship data.

[0082] The sub-mapping relationship data corresponding to the data groups that have not been updated in the data group set has been determined during the generation of the mapping relationship data.

[0083] For each sub-mapping relationship data of a data group in the data group set (including updated and unupdated data groups), keywords that have a mapping relationship with the same standard term are combined to generate new mapping relationship data.

[0084] This implementation provides an incremental update method for mapping relationship data, which combines the sub-mapping relationship data corresponding to the updated data group with the sub-mapping relationship data corresponding to the unupdated data group in the data group set. This reduces the data processing pressure during the data update process while ensuring the accuracy of the mapping relationship data.

[0085] In some optional implementations of this embodiment, the execution entity can perform the second step described above to obtain the mapping relationship data in the following manner: First, by combining the updated sub-mapping relationship data corresponding to the data group and the unupdated sub-mapping relationship data corresponding to the data group in the data group set, the merged data is obtained.

[0086] For each sub-mapping relationship data of a data group (including updated and unupdated data groups) in the data group set, keywords that have a mapping relationship with the same standard term are combined to generate merged data.

[0087] Then, for standard terms with multiple keywords in the merged data, a preset filtering rule is used to determine the target words from the multiple keywords in order to generate mapping relationship data.

[0088] For example, a preset filtering rule indicates that a preset number of keywords will be retained when co-occurrence frequencies are sorted from highest to lowest. Another example is a preset filtering rule that indicates that a preset number of keywords will be retained if their co-occurrence frequency with standard terms is greater than a preset frequency threshold, and which are also sorted from highest to lowest co-occurrence frequency.

[0089] In this implementation, the merged data is filtered based on preset filtering rules, which further improves the effectiveness of the mapping relationship data and ensures the accuracy of the mapping relationship between standard terms and keywords.

[0090] In some optional implementations of this embodiment, the execution entity can perform the data filtering process based on preset filtering rules in the following manner: First, for standard terms with multiple keywords in the merged data, a preset filtering rule is used to determine the target words from the multiple keywords, resulting in initial relational data dominated by standard terms.

[0091] In this implementation, the preset filtering rules are as described in the example above, and will not be repeated here.

[0092] In initial relational data dominated by standard terms, the mapping relationship between standard terms and keywords is generally one-to-many or one-to-one.

[0093] Then, the standard terms and keywords in the initial relational data are transformed into primary and secondary terms to obtain mapping relational data dominated by keywords.

[0094] The approach will shift from being primarily based on standard terms to being primarily based on keywords. For example, the one-to-many or one-to-one mapping relationship between standard terms and keywords will be transformed into a one-to-many or one-to-one mapping relationship between keywords and standard terms.

[0095] In this implementation, by transforming the primary and secondary mapping relationships, we obtain mapping relationship data dominated by keywords. This helps to improve the convenience and efficiency of the data processing process of determining the standard terms corresponding to the keywords in the query request from the mapping relationship data.

[0096] In some optional implementations of this embodiment, the execution entity can perform the data filtering process based on preset filtering rules in the following ways: First, based on the co-occurrence characteristics of each keyword with standard terms, initial screening words are determined from multiple keywords.

[0097] For example, keywords whose co-occurrence frequency with standard terms is greater than a preset frequency threshold are retained as initial screening keywords. Another example is retaining a preset number of keywords whose co-occurrence frequency is sorted from highest to lowest as initial screening keywords.

[0098] Then, based on the parts of speech of the initial screening words and the standard terms, the words to be screened again are determined from the initial screening words.

[0099] The part of speech is, for example, verb or noun. For multiple initial screening words corresponding to the standard term, the initial screening words with the same part of speech as the standard term are retained as the second screening words.

[0100] Finally, a large-scale artificial intelligence model is used to determine the target words from the re-screened words.

[0101] The standard terms and re-filtered words with mapping relationships are input into the AI ​​big data model. Based on its powerful knowledge base, the AI ​​big data model determines the target words from the re-filtered words.

[0102] This implementation provides a three-level filtering mechanism based on co-occurrence features, part-of-speech tags, and a large model, ensuring the accuracy of the data filtering process.

[0103] In some optional implementations of this embodiment, the execution entity can perform the above-mentioned data filtering operation based on the artificial intelligence big model in the following way: using the artificial intelligence big model, the target word is determined from the re-filtered words according to the preset filtering prompt words.

[0104] The preset filter suggestions are obtained by adjusting the original filter suggestions based on the feedback data of the target object to the reply data by the large artificial intelligence model.

[0105] As an example, the preset filter prompts are adjusted as follows: First, collect feedback data from the target audience on the response data, including explicit evaluations (such as "inaccurate keyword matching" and "unreasonable standard term replacement") and behavioral feedback (such as modification of the response content and repeated submission of similar queries). The feedback data is then categorized and labeled. For example, "keywords missing core meaning" is labeled as "insufficient filtering accuracy," and "sentences are not fluent after term replacement" is labeled as "semantic coherence problem."

[0106] Then, the core issues in the feedback data are extracted and transformed into optimization directions for the original filtering suggestions. For example, if the feedback focuses on "keywords not matching standard terms," ​​the optimization direction is to "enhance the semantic matching constraints between keywords and standard terms"; if the feedback shows that "some industry-specific keywords are mistakenly filtered," the optimization direction is to "add exception rules for industry terms."

[0107] Next, the optimization direction is translated into specific prompt word adjustment instructions. For example, for "enhancing semantic matching constraints", the instruction is generated to "when filtering keywords, the semantic similarity with standard terms should be calculated, and only words with similarity higher than the threshold should be retained"; for "adding industry exception rules", the instruction is generated to "if the keyword is a common expression in a specific industry (such as 'sore throat' in the medical field corresponding to 'pharyngitis'), it should still be retained even if the literal difference is large".

[0108] Then, the adjustment instructions are embedded into the original filter prompts to form candidate preset filter prompts. The artificial intelligence big data model is then invoked to simulate filtering of historical mapping relationship data using the candidate prompts to verify the adjustment effect.

[0109] Finally, if the simulated filtering results meet the optimization expectations of the feedback data, the candidate suggestions are determined as the final preset filtering suggestions; if there are still deviations (such as insufficient number of keywords due to over-filtering), the above steps are repeated, and the instructions are further fine-tuned based on the new feedback data until the generated preset filtering suggestions can solve the core problems of the target audience.

[0110] In this implementation, the large-scale artificial intelligence model determines the target word from the re-filtered words based on the preset filtering prompts that are iteratively optimized, which helps to continuously ensure the accuracy of the mapping relationship data.

[0111] In some optional implementations of this embodiment, the terminology description data is code data in the code repository document, and the non-terminology description data is comment data in the code repository document.

[0112] In this implementation, the aforementioned execution entity can perform the first step described above in the following manner to determine the sub-mapping relationship data corresponding to the updated data group: First, determine the code names and comment data for adjacent code names in the code repository documentation.

[0113] Code repository documentation is a comprehensive textual description and explanation of a code repository (a collection of code resources including source code, functions, classes, modules, interfaces, configuration files, etc.). It aims to help developers understand the code repository's design philosophy, functional structure, usage methods, technical details, and maintenance rules. Code names and adjacent comment data are identified from the code data in the code repository documentation.

[0114] Then, identify the keywords in the code terms represented by the code names and the comment data.

[0115] For each code name, preprocessing is performed to obtain the code term it represents. For example, the code name "getUserInfo" represents the code term "User Info (user information)", and the code name "updateOrderStatus" represents the code term "Order Status (order status)".

[0116] For annotated data, natural language understanding techniques can be used to extract keywords.

[0117] Finally, based on the co-occurrence characteristics of code terms and keywords, the sub-mapping relationship data corresponding to the code library documents is determined.

[0118] The co-occurrence characteristics of code terms and keywords refer to the association between specific code terms and keywords appearing simultaneously in the same context within a codebase document. It includes the following dimensions of characteristics: Contextual relevance: The two terms often appear in text fragments describing the same function, operation, or concept. For example, in code comments, the code term "object-oriented programming" may appear alongside keywords such as "object-based development" and "class and instance design," all explaining the same programming paradigm.

[0119] Complementarity: Code terminology ensures standardized expression, while keywords reflect the flexibility of usage scenarios. Co-occurrence fully covers both professional expressions and popular understanding. For example, in debugging documentation, "Exception Handling" co-occurs with "error capture" and "try-catch usage." The former is a standard term, while the latter is a keyword commonly used by developers, both pointing to the same code logic.

[0120] Frequency dependence: The higher the co-occurrence frequency, the stronger the semantic relationship between the two.

[0121] This implementation provides a targeted method for generating sub-mapping relationship data in the context of code repository documents, thereby improving the accuracy of sub-mapping relationship data generated based on code repository documents.

[0122] In some optional implementations of this embodiment, the execution entity can identify the code terms represented by the code name in the following way: First, determine the initial words represented by the code name based on the naming convention.

[0123] Code names can be categorized using various naming conventions, such as camelCase and underscores. CamelCase distinguishes word boundaries using uppercase and lowercase letters, named for its resemblance to the humps of a camel. It includes both lowercase and uppercase camelCase. In lowercase camelCase, the first word of the code name is all lowercase, and the first letter of each subsequent word is capitalized. Examples include `userName` (username), `getUserInfo` (get user information), and `isValidData` (check if data is valid). In uppercase camelCase, the first letter of every word in the code name is capitalized. Examples include `UserAccount` (user account class), `OrderService` (order service class), and `DataProcessing` (data processing module).

[0124] Underscore naming conventions use underscores "_" as word separators, and all letters are typically lowercase (uppercase can be used according to conventions, such as for constants). Words are connected by a single underscore, clearly distinguishing each component. Examples include user_name (username), get_user_info (get user information), and is_valid_data (whether the data is valid). For constants, all uppercase may be used, such as MAX_LENGTH (maximum length) and DB_CONNECTION (database connection).

[0125] Based on the naming convention, the code name can be broken down into words, bigrams, trigrams, and initial words.

[0126] Then, stop words are removed from the initial words to obtain the filtered words.

[0127] Commonly used stop words include get, set, add, remove, delete, update, create, find, interface, main, controller, service, impl, and dto.

[0128] For example, the initial term for `get_user_info` is "get user info". Removing the stop word "get" yields the filtered term "user info".

[0129] Finally, code terms are determined from the filtered words based on their parts of speech.

[0130] As an example, filter words that are either verbs or nouns can be used as code terms.

[0131] This implementation provides a method for generating code terminology, which improves the accuracy of code terminology by combining code features such as code name, stop words, and parts of speech.

[0132] In some optional implementations of this embodiment, the aforementioned execution entity may also perform the following operations: optimize the retrieval method and intent recognition model used to obtain retrieval results based on the feedback data using a large-scale artificial intelligence model. The feedback data includes query requests, retrieval results, mapping relationship data, response data, and feedback data from the target object to the response data.

[0133] As an example, the intent recognition model can be optimized as follows: First, extract the query requests, response data and target object feedback (such as "misunderstanding the query intent" and "incorrect intent classification") from the return data, construct an error sample set for intent recognition, and label the intent types that the model misjudged (such as misjudging "consultation operation steps" as "query function parameters").

[0134] Then, the big AI model is used to perform semantic parsing on the query requests in the error sample set. Combined with the standard terms in the mapping relationship data, the semantic bias behind the misjudgment is mined (such as the model failing to recognize the association between "how to use" and "operation steps"), and targeted fine-tuning samples are generated (such as binding "how to use XX function" with "operation intention").

[0135] Next, the intent recognition model is incrementally trained using fine-tuned samples, with a focus on enhancing its ability to distinguish easily confused intents. Finally, the optimized intent recognition model is applied to new query requests, and the effect is verified by combining the search results and response data in the return data. If the intent recognition accuracy improves and the target object's feedback improves, the model optimization is complete; otherwise, fine-tuning is repeated based on new erroneous samples until the model performance is stable and meets the target.

[0136] As another example, the algorithm can be optimized as follows: First, collect the query requests, search results, mapping relationship data and the target object's feedback on the response data from the return data, and filter out the samples with feedback of "low relevance of search results" and "missing key information" to clarify the optimization direction of the search method (such as insufficient search term coverage and strategy matching deviation).

[0137] Then, the AI ​​big data model is called to analyze the query requests and mapping relationship data in the sample, identify high-frequency keywords that are not covered by existing search terms (such as expressions that users often use but are not included in the mapping relationship), and expand the search term library based on co-occurrence relationships (such as increasing the association strength between "fast charging" and "fast charging" to ensure synchronous retrieval).

[0138] Next, regarding the retrieval strategy, we analyzed the samples with "redundant retrieval results" in the feedback, extracted the invalid filtering conditions (such as over-reliance on a certain category label leading to an overly broad scope), and generated new hierarchical filtering rules through a large model (such as first narrowing the scope with core terms, and then using related keywords for precise matching).

[0139] Finally, the optimized search terminology and strategy are used to validate historical samples. If the target audience's satisfaction with the response data increases (e.g., the proportion of "highly relevant" feedback increases), then it is identified as the new search method. If there is still a deviation, the above steps are repeated until the matching accuracy between the search results and the query request reaches the expected level.

[0140] In this implementation, during the data generation process based on a large model, the retrieval method and intent recognition model are iteratively optimized based on the reflux data, ensuring the accuracy of the retrieval results and the intent recognition of the query request, which helps to continuously improve the accuracy of the response data.

[0141] Continue to refer to Figure 4 The illustration shows a schematic flow 400 of yet another embodiment of the large-model-based data generation method according to this disclosure. Flow 400 includes the following steps: Step 401: Determine the standard terms corresponding to the keywords in the query request from the mapping relationship data.

[0142] Mapping relationship data represents the mapping relationship between keywords and standard terms.

[0143] Step 402: Replace the keywords in the query request with standard terms to obtain the initial adjusted request.

[0144] Step 403: Based on the association characteristics of the query request, determine the query intent of the initially adjusted request.

[0145] Step 404: Using a large-scale artificial intelligence model, the query request is reconstructed based on the query intent to generate an adjusted query request.

[0146] Step 405: Determine the retrieval task based on the adjusted query request.

[0147] The retrieval task includes the detection target, retrieval type, search terms, and retrieval strategy.

[0148] Step 406: Execute the search task and obtain the search results.

[0149] Step 407: Using a large-scale artificial intelligence model, generate response data for the query request based on the search results.

[0150] The process 400 of the data generation method based on the large model in this embodiment, compared with the above process 200, specifically describes the process of adjusting the query request and determining the retrieval results, which further improves the accuracy and comprehensiveness of the expression of the adjusted query request, helps the artificial intelligence large model understand the query intent of the target object, and improves the accuracy of the response data obtained by processing the query request based on the artificial intelligence large model.

[0151] Continue to refer to Figure 5 The illustration shows a schematic flow 500 of an embodiment of a method for generating and updating mapping relationship data according to the present disclosure. Flow 500 includes the following steps: Step 501: For the terminology description data and non-terminology description data in the data group set, based on the co-occurrence characteristics between standard terms in the terminology description data and keywords in the non-terminology description data, determine the sub-mapping relationship data that represents the mapping relationship between standard terms and keywords in the data group.

[0152] Step 502: Combine the sub-mapping relationship data of each data group in the data group set to generate mapping relationship data.

[0153] Step 503: In response to the data update condition being met, for the terminology description data and non-terminology description data in the updated data group in the data group set, the sub-mapping relationship data corresponding to the updated data group is determined based on the co-occurrence characteristics between standard terms in the terminology description data and keywords in the non-terminology description data.

[0154] Step 504: Combine the updated sub-mapping relationship data corresponding to the data group with the unupdated sub-mapping relationship data corresponding to the data group in the data group set to obtain the merged data.

[0155] Step 505: For standard terms with multiple keywords in the merged data, determine the initial screening words from the multiple keywords based on the co-occurrence characteristics between each keyword and the standard term.

[0156] Step 506: Determine the re-screening words from the initial screening words based on the parts of speech of the initial screening words and the standard terms.

[0157] Step 507: Using a large artificial intelligence model, target words are identified from the re-screened words to obtain initial relational data dominated by standard terms.

[0158] Step 508: Perform a primary-secondary transformation on the standard terms and keywords in the initial relational data to obtain mapping relational data dominated by keywords.

[0159] The mapping relationship data generation and update method 500 in this embodiment specifically describes the generation and update process of the mapping relationship data, which can ensure the accuracy and comprehensiveness of the mapping relationship data.

[0160] Continue to refer to Figure 6 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of a data generation device based on a large model. This system embodiment is similar to... Figure 2 Corresponding to the method embodiments shown, the system can be specifically applied to various electronic devices.

[0161] like Figure 6 As shown, the data generation device 600 based on a large model includes: a terminology mapping unit 601, configured to determine the standard terms corresponding to the keywords in the query request from the mapping relationship data, wherein the mapping relationship data represents the mapping relationship between the keywords and the standard terms; a request adjustment unit 602, configured to adjust the query request according to the standard terms to generate an adjusted query request; a data retrieval unit 603, configured to perform data retrieval according to the adjusted query request to obtain retrieval results; and a data generation unit 604, configured to use an artificial intelligence large model to generate response data for the query request based on the retrieval results.

[0162] In some optional implementations of this embodiment, the request adjustment unit 602 is further configured to: use a pre-trained intent recognition model to determine the query intent of the query request based on standard terminology; and use an artificial intelligence big data model to reconstruct the query request based on the query intent to generate an adjusted query request.

[0163] In some optional implementations of this embodiment, the request adjustment unit 602 is further configured to: replace the keywords in the query request with standard terms to obtain an initial adjusted request; and determine the query intent of the initial adjusted request by combining the association features of the query request.

[0164] In some optional implementations of this embodiment, the data retrieval unit 603 is further configured to determine a retrieval task based on the adjusted query request, wherein the retrieval task includes a detection target, a retrieval type, search terms, and a retrieval strategy; execute the retrieval task, and obtain retrieval results.

[0165] In some optional implementations of this embodiment, the above-mentioned apparatus further includes a data construction unit (not shown in the figure), configured to generate mapping relationship data in the following manner: for terminology description data and non-terminology description data in the data group of the data group set, based on the co-occurrence characteristics between standard terms in the terminology description data and keywords in the non-terminology description data, determine sub-mapping relationship data representing the mapping relationship between standard terms and keywords in the data group; combine the sub-mapping relationship data of each data group in the data group set to generate mapping relationship data.

[0166] In some optional implementations of this embodiment, the above-mentioned apparatus further includes a data update unit (not shown in the figure), configured to update the mapping relationship data in the following manner: in response to the achievement of the data update condition, for the term description data and non-term description data in the updated data group in the data group set, determine the sub-mapping relationship data corresponding to the updated data group based on the co-occurrence characteristics between the standard terms in the term description data and the keywords in the non-term description data; and generate mapping relationship data by combining the sub-mapping relationship data corresponding to the updated data group and the sub-mapping relationship data corresponding to the unupdated data group in the data group set.

[0167] In some optional implementations of this embodiment, the data update unit is further configured to: combine the corresponding sub-mapping relationship data of the updated data group with the sub-mapping relationship data of the unupdated data group in the data group set to obtain merged data; for standard terms with multiple keywords in the merged data, a preset filtering rule is used to determine the target word from the multiple keywords in order to generate mapping relationship data.

[0168] In some optional implementations of this embodiment, the data update unit is further configured to: for standard terms with multiple keywords in the merged data, use preset filtering rules to determine the target word from the multiple keywords to obtain initial relational data dominated by standard terms; perform primary and secondary transformation on the standard terms and keywords in the initial relational data to obtain mapping relational data dominated by keywords.

[0169] In some optional implementations of this embodiment, the data update unit is further configured to: determine initial screening words from multiple keywords based on the co-occurrence characteristics between each keyword and the standard term; determine rescreening words from the initial screening words based on the parts of speech of each of the initial screening words and the standard term; and determine the target word from the rescreening words using an artificial intelligence big data model.

[0170] In some optional implementations of this embodiment, the data update unit is further configured to: use an artificial intelligence big data model to determine the target word from the re-filtered words according to the preset filter prompt words, wherein the preset filter prompt words are obtained by the artificial intelligence big data model by adjusting the original filter prompt words according to the feedback data of the target object on the reply data.

[0171] In some optional implementations of this embodiment, the terminology description data is code data in the code library document, the non-terminology description data is comment data in the code library document, and the data update unit is further configured to: determine the code name and the comment data of neighboring code names in the code library document; identify the code terms represented by the code name and the keywords in the comment data; and determine the sub-mapping relationship data corresponding to the code library document based on the co-occurrence characteristics of the code terms and keywords.

[0172] In some optional implementations of this embodiment, the data update unit is further configured to: determine the initial word represented by the code name according to the naming method of the code name; remove stop words from the initial word to obtain filter words; and determine code terms from the filter words according to the part of speech.

[0173] This embodiment provides a data generation device based on a large model. It determines the standard terms corresponding to keywords in a query request from mapping relationship data, where the mapping relationship data represents the mapping relationship between keywords and standard terms. The query request is adjusted based on the standard terms to generate an adjusted query request. Data retrieval is performed based on the adjusted query request to obtain retrieval results. An artificial intelligence large model is used to generate response data for the query request based on the retrieval results. By determining the standard terms for keywords in the query request through mapping relationship data to adjust the query request, the accuracy and comprehensiveness of the adjusted query request are improved. This helps the artificial intelligence large model understand the query intent of the target object and improves the accuracy of the response data obtained from processing the query request based on the artificial intelligence large model.

[0174] According to embodiments of this disclosure, this disclosure also provides an electronic device, the electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to implement the large model-based data generation method described in any of the above embodiments.

[0175] According to embodiments of this disclosure, this disclosure also provides a readable storage medium storing computer instructions that enable a computer to implement the large model-based data generation method described in any of the above embodiments when executed.

[0176] This disclosure provides a computer program product that, when executed by a processor, can implement the data generation method based on a large model as described in any of the above embodiments.

[0177] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0178] like Figure 7 As shown, device 700 includes a computing unit 701, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 702 or a computer program loaded into random access memory (RAM) 703 from storage unit 708. The RAM 703 may also store various programs and data required for the operation of device 700. The computing unit 701, ROM 702, and RAM 703 are interconnected via bus 704. Input / output (I / O) interface 705 is also connected to bus 704.

[0179] Multiple components in device 700 are connected to I / O interface 705, including: input unit 706, such as keyboard, mouse, etc.; output unit 707, such as various types of monitors, speakers, etc.; storage unit 708, such as disk, optical disk, etc.; and communication unit 709, such as network card, modem, wireless transceiver, etc. Communication unit 709 allows device 700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0180] The computing unit 701 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as the large model-based data generation method. For example, in some embodiments, the large model-based data generation method can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed on device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of the large model-based data generation method described above can be performed. Alternatively, in other embodiments, the computing unit 701 can be configured to perform the large model-based data generation method by any other suitable means (e.g., by means of firmware).

[0181] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0182] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable model-based data generation device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0183] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0184] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0185] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0186] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, also known as cloud computing servers or cloud hosts, which are hosting products within the cloud computing service system to address the management difficulties and weak business scalability inherent in traditional physical hosts and Virtual Private Servers (VPS) services; they can also be servers for distributed systems or servers incorporating blockchain technology.

[0187] According to the technical solution of this disclosure, a data generation method and apparatus based on a large model are provided. The method determines the standard terms corresponding to keywords in a query request from mapping relationship data, wherein the mapping relationship data represents the mapping relationship between keywords and standard terms. The query request is adjusted according to the standard terms to generate an adjusted query request. Data retrieval is performed based on the adjusted query request to obtain retrieval results. An artificial intelligence large model is used to generate response data for the query request based on the retrieval results. This method improves the accuracy and comprehensiveness of the adjusted query request by determining the standard terms for keywords in the query request through mapping relationship data, which helps the artificial intelligence large model understand the query intent of the target object and improves the accuracy of the response data obtained by processing the query request based on the artificial intelligence large model.

[0188] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution provided in this disclosure can be achieved, and this is not limited herein.

[0189] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A data generation method based on a large model, comprising: The standard terms corresponding to the keywords in the query request are determined from the mapping relationship data, wherein the mapping relationship data represents the mapping relationship between keywords and standard terms; Adjust the query request according to the standard terminology to generate an adjusted query request; Data retrieval is performed based on the adjusted query request to obtain the retrieval results; Using a large-scale artificial intelligence model, response data is generated based on the search results to address the query request.

2. The method according to claim 1, wherein, The step of adjusting the query request according to the standard terminology to generate an adjusted query request includes: A pre-trained intent recognition model is used to determine the query intent of the query request based on the standard terminology. Using the aforementioned large-scale artificial intelligence model, the query request is reconstructed based on the query intent to generate the adjusted query request.

3. The method according to claim 2, wherein, Determining the query intent of the query request based on the standard terminology includes: Replace the keywords in the query request with the standard terms to obtain the initial adjusted request; Based on the association characteristics of the query request, the query intent of the initially adjusted request is determined.

4. The method according to claim 1, wherein, The step of retrieving data based on the adjusted query request and obtaining the retrieval results includes: Based on the adjusted query request, a retrieval task is determined, wherein the retrieval task includes the detection target, retrieval type, retrieval terms, and retrieval strategy; The search task is executed to obtain the search results.

5. The method according to any one of claims 1-4, wherein, The mapping relationship data is generated in the following manner: For terminology description data and non-terminology description data in the data group set, based on the co-occurrence characteristics between standard terms in the terminology description data and keywords in the non-terminology description data, sub-mapping relationship data that characterizes the mapping relationship between standard terms and keywords in the data group is determined, wherein the co-occurrence characteristics characterize the co-occurrence relationship between standard terms and keywords in the data group; The mapping relationship data is generated by combining the sub-mapping relationship data of each data group in the data group set.

6. The method according to claim 5, wherein, The mapping relationship data is updated in the following manner: In response to the achievement of the data update condition, for the term description data and non-term description data in the updated data group in the data group set, the sub-mapping relationship data corresponding to the updated data group is determined based on the co-occurrence characteristics between the standard terms in the term description data and the keywords in the non-term description data; The mapping relationship data is generated by combining the updated sub-mapping relationship data corresponding to the data group and the unupdated sub-mapping relationship data corresponding to the data group in the data group set.

7. The method according to claim 6, wherein, The step of generating the mapping relationship data by combining the updated sub-mapping relationship data corresponding to the data group and the unupdated sub-mapping relationship data corresponding to the data group set includes: By combining the updated sub-mapping relationship data corresponding to the data group and the unupdated sub-mapping relationship data corresponding to the data group in the data group set, the merged data is obtained; For standard terms with multiple keywords in the merged data, a preset filtering rule is used to determine the target word from the multiple keywords in order to generate the mapping relationship data.

8. The method according to claim 7, wherein, For standard terms with multiple keywords in the merged data, a preset filtering rule is used to determine target words from the multiple keywords to generate the mapping relationship data, including: For standard terms with multiple keywords in the merged data, a preset filtering rule is used to determine the target words from the multiple keywords, resulting in initial relational data dominated by standard terms. The standard terms and keywords in the initial relational data are transformed into primary and secondary data to obtain mapping relational data dominated by keywords.

9. The method according to claim 7 or 8, wherein, The step of using preset filtering rules to determine target words from multiple keywords includes: Based on the co-occurrence characteristics between each of the keywords and the standard terms, initial screening words are determined from the multiple keywords; Based on the part of speech of the initial screening words and the standard terms, the rescreening words are determined from the initial screening words; The target word is determined from the re-screened words using the aforementioned large-scale artificial intelligence model.

10. The method according to claim 9, wherein, The process of using the aforementioned large-scale artificial intelligence model to determine the target word from the re-screened words includes: Using the aforementioned large-scale artificial intelligence model, the target word is determined from the re-filtered words based on preset filtering prompts. The preset filtering prompts are obtained by adjusting the original filtering prompts based on the feedback data of the target object to the response data.

11. The method according to claim 5 or 6, wherein, The terminology description data is code data from the codebase document, and the non-terminology description data is comment data from the codebase document. The step of determining the sub-mapping relationship data corresponding to the updated data group based on the co-occurrence characteristics between standard terms in the terminology description data and keywords in the non-terminology description data includes: Determine the code name and the comment data adjacent to the code name in the code repository document; Identify the code terms represented by the code names and the keywords in the comment data; Based on the co-occurrence characteristics of the code terms and keywords, the sub-mapping relationship data corresponding to the code library document is determined.

12. The method according to claim 11, wherein, The identification of the code terms represented by the code name includes: Based on the naming convention of the code name, determine the initial word represented by the code name; Remove stop words from the initial words to obtain the filtered words; The code terms are determined from the filtered words based on their parts of speech.

13. A data generation device based on a large model, comprising: A term mapping unit is configured to determine standard terms corresponding to keywords in a query request from mapping relationship data, wherein the mapping relationship data represents the mapping relationship between keywords and standard terms; The request adjustment unit is configured to adjust the query request according to the standard terminology and generate an adjusted query request. The data retrieval unit is configured to perform data retrieval based on the adjusted query request and obtain retrieval results; The data generation unit is configured to use a large artificial intelligence model to generate response data for the query request based on the search results.

14. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-13.

15. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-13.

16. A computer program product comprising: A computer program that, when executed by a processor, implements the method according to any one of claims 1-13.

Images