Large model cue word generation method and device, equipment and medium

Abstract

The invention belongs to the technical field of artificial intelligence, is suitable for financial scenes and medical scenes, and discloses a large-model cue word generation method, device and equipment and a medium, and the method comprises the steps: obtaining multi-modal data corresponding to a prompt task, and carrying out the data preprocessing of the multi-modal data; inputting the preprocessed multi-modal data into a pre-trained heterogeneous data encoder for cross-modal feature extraction, and generating an initial prompt word set based on the extracted cross-modal features; constructing a structured knowledge graph and an unstructured database associated with the prompt task, and injecting domain knowledge into the initial prompt word based on the structured knowledge graph and the unstructured database to generate a domain enhanced candidate prompt word set; and constructing an intelligent agent framework, inputting the candidate prompt word set into the intelligent agent framework for semantic feature extraction, and generating a target prompt word set according to the extracted semantic features. According to the method, the generation efficiency of the cue word of the large model is effectively improved.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology and is applicable to financial and medical scenarios. In particular, it relates to a method, device, equipment and medium for generating prompt words for a large model. Background Art

[0002] With the rapid development of artificial intelligence technology, natural language processing systems based on large language models have been widely used in specialized fields such as financial investment analysis and medical diagnosis assistance. However, existing prompt word generation methods still face many challenges in complex business scenarios.

[0003] First, traditional methods mainly rely on manual experience to construct static templates, which are difficult to adapt to real-time changing business needs. For example, in the field of financial investment, market conditions may fluctuate at the minute level, and existing static prompt word templates cannot be dynamically adjusted to adapt to rapidly changing investment strategies; in medical scenarios, patients' conditions may evolve rapidly, but fixed templates cannot flexibly adapt to diagnostic needs at different stages. Secondly, most current prompt word generation methods are still based on simple keyword matching or template filling mechanisms, which fail to fully integrate domain knowledge bases and contextual information, resulting in the generated prompt words lacking professional depth and accuracy. In addition, existing methods generally lack personalized adaptation capabilities. For example, in the financial field, different investors have significantly different risk preferences, but existing methods find it difficult to generate targeted prompt words; in medical scenarios, the individual characteristics of patients are not fully incorporated into the prompt word optimization process, resulting in insufficient adaptability of the output diagnostic recommendations.

[0004] In view of this, how to improve the efficiency of generating prompt words for large models is a technical problem that needs to be solved urgently. Summary of the Invention

[0005] The present invention provides a method, device, equipment and medium for generating prompt words of a large model, so as to solve the technical problem of low efficiency in generating prompt words of the existing large model.

[0006] In a first aspect, the present invention provides a method for generating prompt words for a large model, comprising:

[0007] Acquire multimodal data corresponding to the prompt task, and perform data preprocessing on the multimodal data, wherein the multimodal data includes at least text data, voice data, and image data;

[0008] The preprocessed multimodal data is input into a pre-trained heterogeneous data encoder for cross-modal feature extraction, and an initial prompt word set is generated based on the extracted cross-modal features;

[0009] Constructing a structured knowledge graph and an unstructured database associated with the prompt task, and injecting domain knowledge into the initial prompt word set based on the structured knowledge graph and the unstructured database to generate a domain-enhanced candidate prompt word set;

[0010] An intelligent agent framework is constructed, the candidate prompt word set is input into the intelligent agent framework for semantic feature extraction, and a target prompt word set is generated according to the extracted semantic features.

[0011] In a second aspect, the present invention provides a large-model prompt word generation device, comprising:

[0012] a processing module, configured to obtain multimodal data corresponding to the prompt task and perform data preprocessing on the multimodal data, wherein the multimodal data includes at least text data, voice data, and image data;

[0013] An extraction module, configured to input the preprocessed multimodal data into a pre-trained heterogeneous data encoder for cross-modal feature extraction, and generate an initial prompt word set based on the extracted cross-modal features;

[0014] A construction module is used to construct a structured knowledge graph and an unstructured database associated with the prompt task, and inject domain knowledge into the initial prompt word set based on the structured knowledge graph and the unstructured database to generate a domain-enhanced candidate prompt word set;

[0015] The generation module is used to construct an intelligent agent framework, input the candidate prompt word set into the intelligent agent framework to extract semantic features, and generate a target prompt word set based on the extracted semantic features.

[0016] In a third aspect, the present invention provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the method for generating prompt words for the large model are implemented.

[0017] In a fourth aspect, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the method for generating prompt words for the large model.

[0018] The method, device, equipment and medium for generating prompt words of the above-mentioned large model can obtain multimodal data corresponding to the prompt task through the client, and perform data preprocessing on the multimodal data, wherein the multimodal data at least includes text data, voice data and image data; input the preprocessed multimodal data into a pre-trained heterogeneous data encoder for cross-modal feature extraction, and generate an initial prompt word set based on the extracted cross-modal features; construct a structured knowledge graph and an unstructured database associated with the prompt task, and perform data preprocessing on the multimodal data based on the structured knowledge graph and the unstructured database. The initial prompt word set is injected with domain knowledge to generate a domain-enhanced candidate prompt word set; an intelligent agent framework is constructed, the candidate prompt word set is input into the intelligent agent framework for semantic feature extraction, and a target prompt word set is generated based on the extracted semantic features. In the present invention, when generating prompt words for a large model, a domain-enhanced candidate prompt word set can be generated by combining a structured knowledge graph with an unstructured database. The semantic ambiguity and logical coherence of the candidate prompt words are optimized by the intelligent agent framework, which can make the generated target prompt words more accurate, significantly reduce the cost of manual correction, and improve the generation efficiency of prompt words for the large model. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0020] Figure 1 This is a schematic diagram of an application environment of a method for generating prompt words for a large model according to an embodiment of the present invention;

[0021] Figure 2 This is a flow chart of a method for generating prompt words for a large model according to an embodiment of the present invention;

[0022] Figure 3 yes Figure 2 A schematic flow chart of a specific implementation of step S20;

[0023] Figure 4 yes Figure 2 A schematic flow chart of a specific implementation of step S30;

[0024] Figure 5 yes Figure 2 A schematic flow chart of a specific implementation of step S40;

[0025] Figure 6 1 is a structural diagram of a prompt word generating device for a large model in one embodiment of the present invention;

[0026] Figure 7 is a structural diagram of a computer device in one embodiment of the present invention;

[0027] Figure 8 FIG. 2 is another structural diagram of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0029] The method for generating prompt words of a large model provided by the embodiment of the present invention can be applied in the following situations: Figure 1 In the application environment, Figure 1 This is a schematic diagram of an application environment of a method for generating prompt words for a large model in one embodiment of the present invention; wherein the client communicates with the server through a network. The server can obtain multimodal data corresponding to the prompt task through the client and perform data preprocessing on the multimodal data, wherein the multimodal data includes at least text data, voice data and image data; input the preprocessed multimodal data into a pre-trained heterogeneous data encoder for cross-modal feature extraction, and generate an initial prompt word set based on the extracted cross-modal features; construct a structured knowledge graph and an unstructured database associated with the prompt task, and inject domain knowledge into the initial prompt word set based on the structured knowledge graph and the unstructured database to generate a domain-enhanced candidate prompt word set; construct an intelligent agent framework, input the candidate prompt word set into the intelligent agent framework for semantic feature extraction, and generate a target prompt word set based on the extracted semantic features. In the present invention, when generating prompt words for a large model, a domain-enhanced candidate prompt word set can be generated by combining the structured knowledge graph and the unstructured database. The intelligent agent framework optimizes the semantic ambiguity and logical coherence of the candidate prompt words, which can make the generated target prompt words more accurate, significantly reduce the cost of manual correction, and improve the efficiency of generating prompt words for the large model. The present invention is described in detail below through specific examples.

[0030] See also Figure 2 As shown, Figure 2 A flowchart of a method for generating prompt words for a large model according to an embodiment of the present invention is provided. The method for generating prompt words for a large model specifically includes the following steps:

[0031] S10: Obtain multimodal data corresponding to the prompt task and perform data preprocessing on the multimodal data, wherein the multimodal data includes at least text data, voice data, and image data. When generating the prompt words of the large model, the present invention first obtains the multimodal data corresponding to the pre-prompt task, wherein the multimodal data supports multiple data forms such as text, voice, and image, avoiding the limitations caused by a single modality. After obtaining the multimodal data corresponding to the prompt task, it is also necessary to perform data preprocessing on the obtained multimodal data to ensure data quality and reduce subsequent analysis errors. For example, in a financial scenario, a user uploads a voice command "Help me analyze the company's latest financial report" and a PDF screenshot of the financial report. After preprocessing, structured text and table data can be extracted. In a medical scenario, the voice of a doctor describing the patient's symptoms and an electrocardiogram image are uploaded. After preprocessing, the voice data and image data can be converted into analyzable text and digital signals.

[0032] S20: Input the pre-processed multimodal data into the pre-trained heterogeneous data encoder to extract cross-modal features, and generate an initial prompt word set based on the extracted cross-modal features. After the multimodal data is pre-processed in step S10, the present invention also needs to extract cross-modal features from the pre-processed multimodal data. Specifically, Figure 3 As stated, Figure 3 This is a flow chart of a specific implementation of step S20, which specifically includes the following steps S21-S23:

[0033] S21: The preprocessed text data, speech data, and image data are respectively input into the heterogeneous data encoder for feature extraction, thereby obtaining a text feature vector, a speech feature vector, and an image feature vector for the prompt task. Specifically, in an embodiment of the present invention, the text, speech, and image data are respectively processed by the heterogeneous data encoder to obtain a text feature vector, a speech feature vector, and an image feature vector for the prompt task. For example, in a financial scenario, the text data may be the numbers and analytical content in a financial report, and the text feature vector reflects the financial report analysis information; the speech data may be a recording of an investor meeting, and the speech feature vector may reflect the tone and intention of the investor meeting; the image data may be a data chart in a financial report, and the image feature vector may reflect the changing trend of the data in the chart. In a medical scenario, the text data may be a patient's medical history recorded by a doctor, and the text feature vector may reflect the patient's medical history and clinical symptoms; the speech data may be a doctor's oral diagnosis, and the speech feature vector may capture the doctor's tone, judgment, and intonation; the image data may be an X-ray or electrocardiogram, and the image feature vector may reflect the diagnostic information that affects the examination.

[0034] S22: Assign weights to the text feature vector, speech feature vector, and image feature vector of the prompt task based on the attention mechanism, and perform cross-modal fusion on the text feature vector, speech feature vector, and image feature vector based on the assigned weights to obtain a feature vector after cross-modal fusion. Specifically, in an embodiment of the present invention, the importance of data of different modalities can be assigned weights through the attention mechanism to ensure that the most critical information is highlighted. For example, in a financial scenario, when analyzing financial reports, investor conference call recordings, and financial report charts, text feature vectors are more important. In a medical scenario, when determining the cause of a patient with chest pain, the doctor's oral diagnosis and the patient's medical history may provide clinical information, while the electrocardiogram image provides specific physiological data, which is more important.

[0035] S23: Input the cross-modally fused feature vector into the pre-trained large language model to generate an initial prompt word set that matches the prompt task. Specifically, in an embodiment of the present invention, by inputting the cross-modally fused feature vector into the large language model, a more accurate, rich and relevant initial prompt word set can be generated. For example, in a financial scenario, after inputting the fused feature vector, the large language model generates initial prompt words: "Please analyze the company's revenue and net profit in 2025", and these prompt words will guide the subsequent analysis generation. For another example, in a medical scenario, after inputting the fused feature vector, the large language model generates initial prompt words: "Possible causes of the patient's chest pain symptoms: differentiation between acute coronary syndrome and pulmonary embolism", and these prompt words will help doctors quickly form diagnostic ideas and effectively improve the efficiency of clinical diagnosis.

[0036] S30: Constructing a structured knowledge graph and an unstructured database associated with the prompt task, and injecting domain knowledge into the initial prompt word set based on the structured knowledge graph and the unstructured database to generate a domain-enhanced candidate prompt word set. In this embodiment of the present invention, by injecting domain knowledge into the unstructured knowledge database, the professionalism of the generated prompt words can be ensured. Specifically, Figure 4 As stated, Figure 4 This is a flow chart of a specific implementation of step S30, which specifically includes the following steps S31-S33:

[0037] S31: Construct a structured knowledge graph based on the prompt task, match the keywords of the initial prompt word set with the entity nodes in the knowledge graph, and obtain the target entities associated with the keywords in the initial prompt word set and the association paths between the target entities. Specifically, in an embodiment of the present invention, linking the keywords in the initial prompt words to the entity nodes in the knowledge graph can enhance the professionalism of the prompt words, and by mining the path relationship between entities, it helps to generate prompt words with more logical coherence and contextual reasoning capabilities. For example, in a medical scenario, the initial prompt word is "assess the possible causes of the patient's chest pain", and the medical knowledge graph contains entities such as "chest pain, myocardial infarction" and the relationship is "symptoms, disease"; the target entity and the association path between the target entities are: "chest pain, myocardial infarction, hypertension", then the clinical prompt can be derived: "Assess the risk of myocardial infarction in combination with a history of hypertension."

[0038] S32: Performing a text paragraph search on the unstructured database of the prompt task, calculating the semantic similarity between the text paragraphs in the unstructured database and the initial prompt word set, and screening out text paragraphs whose semantic similarity is greater than a preset semantic similarity threshold. Specifically, in an embodiment of the present invention, highly relevant paragraphs can be extracted from actual text. By calculating the semantic similarity between the text paragraphs in the unstructured database and the initial prompt word set, corpus truly relevant to the task can be retained, thereby enhancing the contextual richness of the generated content.

[0039] S33: Fusing the association paths between the target entities with the selected text paragraphs to generate a domain-enhanced candidate prompt word set. Specifically, in this embodiment of the present invention, by fusing the structural logic in the graph with the contextual information of the unstructured text, the generated prompt words are both accurate and rich in practical context. Specifically, step S33 includes:

[0040] S331: Perform graph attention encoding on the association paths between target entities to generate a structured knowledge feature vector. Specifically, in this embodiment of the present invention, a graph attention network is used to encode the target entities and their paths in the knowledge graph. This encoding method not only focuses on the entities themselves but also dynamically adjusts the importance of adjacent entities through an attention mechanism, enhancing the understanding of the "critical path" in structural knowledge. The output is a vector representing structured knowledge that captures the semantic, causal, or logical relationships between entities.

[0041] S332: Semantically encode the filtered text paragraphs to generate an unstructured knowledge feature vector. Specifically, in an embodiment of the present invention, a pre-trained language model can be used to semantically encode the text paragraphs filtered from the unstructured database, and the obtained feature vector contains the context information and industry domain knowledge expressed in natural language. For example, in a medical scenario, the text paragraph is "The latest research shows that if patients with chest pain have high troponin levels, acute myocardial infarction should be considered immediately", which can be encoded to form an unstructured knowledge feature vector, carrying the latest clinical recommendations, and enhancing the timeliness of prompt word generation and real clinical guidance.

[0042] S333: Based on the attention weights, the structured knowledge feature vector and the unstructured knowledge feature vector are weightedly concatenated, and the weighted concatenated feature vector is input into the prompt word generator to generate a domain-enhanced candidate prompt word set. Specifically, in this embodiment of the present invention, the structured knowledge feature vector in S331 and the unstructured knowledge feature vector in S332 can be weightedly concatenated using an attention mechanism, and the fused vector is input into the prompt word generation module to generate a domain-enhanced candidate prompt word set that is more professional and timely.

[0043] S40: Construct an intelligent agent framework, input the candidate prompt word set into the intelligent agent framework for semantic feature extraction, and generate a target prompt word set based on the extracted semantic features. Figure 5 As stated, Figure 5 yes Figure 2 A flow chart of a specific implementation of step S40 in FIG. 4 includes the following steps S41-S42:

[0044] S41: Input the candidate prompt word set into the intelligent agent framework to extract the user intent, knowledge domain, and semantic constraints of the prompt task. Specifically, in an embodiment of the present invention, by analyzing the candidate prompt word set, the intelligent agent can extract the user's core needs and intentions. For example, in a financial scenario, the user's intention may be to analyze financial reports, predict market trends, or assess risks; in a medical scenario, the user's intention may be to diagnose a disease, develop a treatment plan, or assess health risks. The intelligent agent framework not only extracts the intention, but also determines the knowledge domain involved to ensure that the scope of execution of the task is clear and the domain involved is highly professional. In addition, it is necessary to extract the semantic constraints implicit in the user in the task, such as time range, data requirements, analysis depth, etc., to avoid generating irrelevant or overly broad prompt words. S42: Construct a prompt word template generator based on the extracted user intent, knowledge domain, and semantic constraints of the prompt task, and generate the target prompt word based on the constructed prompt word model generator. Specifically, in this embodiment of the present invention, the agent constructs a template that automatically generates target prompts based on information extracted from user intent, knowledge domains, and semantic constraints. This template, based on the needs of the professional domain, ensures that the generated prompts are both accurate and meet task requirements. For example, in a financial scenario, the template automatically generates an analysis framework based on the characteristics of tasks such as financial analysis and market forecasting; in a medical scenario, the template automatically generates appropriate prompts based on diagnostic and treatment requirements.

[0045] In one embodiment of the present invention, after the target prompt word is generated by the constructed model generator, the method further includes: obtaining explicit and implicit user feedback data for the target prompt word, extracting a feedback feature vector based on the explicit and implicit feedback data of the target prompt word; dynamically adjusting the prompt word template generator based on the extracted feedback feature vector of the target prompt word, and updating the target prompt word based on the adjusted prompt word template generator. Specifically, in this embodiment of the present invention, explicit feedback data refers to feedback directly provided by users, such as ratings, clicks, and text reviews, which can clearly reflect the user's satisfaction with the prompt word generation results and task requirements. Implicit feedback data refers to user behavior data, such as whether the user takes further action, dwell time, and click path, which reflects the user's potential interest in and actual needs for the prompt word. By extracting feature vectors from explicit and implicit feedback, user preferences and needs can be quantified. Furthermore, by inputting the feedback feature vectors, the prompt word template generator can dynamically adjust the template based on the user's actual needs and preferences. For example, in a financial scenario, the prompt word template generator can focus on analyzing certain features based on user preferences. In medical scenarios, based on user feedback, the prompt word template generator can dynamically adjust the details of the diagnostic suggestions to ensure that the pathologies or treatment plans that the user is concerned about are covered.

[0046] In one embodiment of the present invention, dynamically adjusting a prompt word template generator based on the extracted feedback feature vectors of a target prompt word and updating the target prompt word based on the adjusted prompt word template generator includes: calculating the dynamic weight of each feedback feature vector of the target prompt word based on the feature distribution of the extracted feedback feature vectors of the target prompt word; dynamically adjusting the keywords and sentence structure in the prompt word template generator based on the calculated dynamic weights of each feedback feature vector; and updating the target prompt word based on the adjusted prompt word template generator. Specifically, in this embodiment of the present invention, the calculated dynamic weights of the feedback feature vectors can not only adjust the emphasis of the keywords, but also adjust the sentence structure of the generated prompt word. For example, in a financial scenario, when generating prompt words for market analysis, adjustments can be made to the sentence structure to make it focus more on market dynamics and risk control rather than simply profit. In a medical scenario, the sentence structure can also be adjusted to make the generated prompt word more focused on emergency treatment.

[0047] It can be seen that in the above scheme, when generating prompt words for a large model, a domain-enhanced set of candidate prompt words can be generated by combining structured knowledge graphs and unstructured databases. By optimizing the semantic ambiguity and logical coherence of the candidate prompt words through the intelligent agent framework, the generated target prompt words can be made more accurate, significantly reducing the cost of manual correction and improving the efficiency of generating prompt words for large models.

[0048] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0049] In one embodiment, a large model prompt word generation device is provided, which corresponds to the large model prompt word generation method in the above embodiment. Figure 6 As shown, Figure 6 6 is a schematic diagram of a large model prompt word generation device according to an embodiment of the present invention. The large model prompt word generation device includes a processing module 61, an extraction module 62, a construction module 63, and a generation module 64. The functional modules are described in detail as follows:

[0050] A processing module 61 is configured to obtain multimodal data corresponding to the prompt task and perform data preprocessing on the multimodal data, wherein the multimodal data includes at least text data, voice data, and image data;

[0051] An extraction module 62 is configured to input the pre-processed multimodal data into a pre-trained heterogeneous data encoder to extract cross-modal features, and generate an initial prompt word set based on the extracted cross-modal features;

[0052] A construction module 63 is configured to construct a structured knowledge graph and an unstructured database associated with the prompt task, and inject domain knowledge into the initial prompt word set based on the structured knowledge graph and the unstructured database to generate a domain-enhanced candidate prompt word set;

[0053] The generation module 64 is used to construct an intelligent agent framework, input the candidate prompt word set into the intelligent agent framework to extract semantic features, and generate a target prompt word set based on the extracted semantic features.

[0054] In one embodiment, the extraction module 62 is specifically configured to:

[0055] Inputting the preprocessed text data, speech data and image data into the heterogeneous data encoder respectively for feature extraction to obtain a text feature vector, a speech feature vector and an image feature vector of the prompt task;

[0056] Based on the attention mechanism, weights are assigned to the text feature vector, speech feature vector, and image feature vector of the prompt task, and based on the assigned weights, the text feature vector, speech feature vector, and image feature vector are cross-modally fused to obtain a cross-modally fused feature vector.

[0057] The cross-modal fused feature vector is input into a pre-trained large language model to generate an initial prompt word set that matches the prompt task.

[0058] In one embodiment, the construction module 63 is specifically configured to:

[0059] Constructing a structured knowledge graph based on the prompt task, matching the keywords of the initial prompt word set with entity nodes in the knowledge graph, and obtaining target entities associated with the keywords in the initial prompt word set and association paths between target entities;

[0060] Performing a text paragraph search on the unstructured database of the prompt task, and calculating the semantic similarity between the text paragraphs in the unstructured database and the initial prompt word set, and screening out text paragraphs whose semantic similarity is greater than a preset semantic similarity threshold;

[0061] The association paths between the target entities are fused with the screened text paragraphs to generate a domain-enhanced candidate prompt word set.

[0062] In one embodiment, the construction module 63 is specifically configured to:

[0063] Perform graph attention encoding on the association paths between target entities to generate structured knowledge feature vectors;

[0064] Semantically encode the selected text paragraphs to generate unstructured knowledge feature vectors;

[0065] Based on the attention weight, the structured knowledge feature vector and the unstructured knowledge feature vector are weightedly spliced, and the weighted spliced ​​feature vector is input into the prompt word generator to generate a domain-enhanced candidate prompt word set.

[0066] In one embodiment, the generating module 64 is specifically configured to:

[0067] The candidate prompt word set is input into the intelligent agent framework to extract the user intention, knowledge domain and semantic constraint conditions of the prompt task.

[0068] A prompt word template generator is constructed based on the extracted user intention, knowledge domain and semantic constraints of the prompt task, and the target prompt word is generated based on the constructed prompt word model generator.

[0069] In one embodiment, the large model prompt word generating device is further used to:

[0070] Obtaining explicit feedback data and implicit feedback data of the user to the target prompt word, and extracting a feedback feature vector based on the explicit feedback data and implicit feedback data of the target prompt word;

[0071] The prompt word template generator is dynamically adjusted based on the feedback feature vector of the extracted target prompt word, and the target prompt word is updated based on the adjusted prompt word template generator.

[0072] In one embodiment, the large model prompt word generating device is further used to:

[0073] Calculate the dynamic weight of each feedback feature vector of the target prompt word based on the feature distribution of the extracted feedback feature vector of the target prompt word;

[0074] Based on the calculated dynamic weights of the feedback feature vectors, the keywords and sentence structures in the prompt word template generator are dynamically adjusted, and the target prompt word is updated based on the adjusted prompt word template generator.

[0075] The present invention provides a device for generating prompt words for a large model. When generating prompt words for a large model, a domain-enhanced candidate prompt word set can be generated by combining a structured knowledge graph with an unstructured database. The semantic ambiguity and logical coherence of the candidate prompt words are optimized through an intelligent agent framework, which can make the generated target prompt words more accurate, significantly reduce the cost of manual correction, and improve the efficiency of generating prompt words for the large model.

[0076] The specific limitations of the large-scale model prompt word generation device can be found in the limitations of the large-scale model prompt word generation method described above and will not be further elaborated here. Each module in the large-scale model prompt word generation device described above can be implemented in whole or in part through software, hardware, or a combination thereof. Each of the modules described above can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a memory in a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0077] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 7 As shown, Figure 7 The present invention is a schematic diagram of the structure of a computer device in one embodiment of the present invention. The computer device includes a processor, a memory, a network interface, and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a large-model prompt word generation method server side.

[0078] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Figure 8 As shown, Figure 8 This is another structural schematic diagram of a computer device in one embodiment of the present invention. The computer device includes a processor, memory, a network interface, a display screen, and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When executed by the processor, the computer program implements the client-side functions or steps of a large-model prompt word generation method.

[0079] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are performed:

[0080] Acquire multimodal data corresponding to the prompt task, and perform data preprocessing on the multimodal data, wherein the multimodal data includes at least text data, voice data, and image data;

[0081] The preprocessed multimodal data is input into a pre-trained heterogeneous data encoder for cross-modal feature extraction, and an initial prompt word set is generated based on the extracted cross-modal features;

[0082] Constructing a structured knowledge graph and an unstructured database associated with the prompt task, and injecting domain knowledge into the initial prompt word set based on the structured knowledge graph and the unstructured database to generate a domain-enhanced candidate prompt word set;

[0083] An intelligent agent framework is constructed, the candidate prompt word set is input into the intelligent agent framework for semantic feature extraction, and a target prompt word set is generated according to the extracted semantic features.

[0084] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0085] Acquire multimodal data corresponding to the prompt task, and perform data preprocessing on the multimodal data, wherein the multimodal data includes at least text data, voice data, and image data;

[0086] The preprocessed multimodal data is input into a pre-trained heterogeneous data encoder for cross-modal feature extraction, and an initial prompt word set is generated based on the extracted cross-modal features;

[0087] Constructing a structured knowledge graph and an unstructured database associated with the prompt task, and injecting domain knowledge into the initial prompt word set based on the structured knowledge graph and the unstructured database to generate a domain-enhanced candidate prompt word set;

[0088] An intelligent agent framework is constructed, the candidate prompt word set is input into the intelligent agent framework for semantic feature extraction, and a target prompt word set is generated according to the extracted semantic features.

[0089] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can be found in the relevant descriptions of the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0090] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchl ink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0091] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0092] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A method for generating prompt words for a large model, characterized in that: include: Acquire multimodal data corresponding to the prompt task, and perform data preprocessing on the multimodal data, wherein the multimodal data includes at least text data, voice data, and image data; The preprocessed multimodal data is input into a pre-trained heterogeneous data encoder for cross-modal feature extraction, and an initial prompt word set is generated based on the extracted cross-modal features; Constructing a structured knowledge graph and an unstructured database associated with the prompt task, and injecting domain knowledge into the initial prompt word set based on the structured knowledge graph and the unstructured database to generate a domain-enhanced candidate prompt word set; An intelligent agent framework is constructed, the candidate prompt word set is input into the intelligent agent framework for semantic feature extraction, and a target prompt word set is generated according to the extracted semantic features.

2. The method for generating prompt words for a large model according to claim 1, characterized in that: The pre-processed multimodal data is input into a pre-trained heterogeneous data encoder for cross-modal feature extraction, and an initial prompt word set is generated based on the extracted cross-modal features, including: Inputting the preprocessed text data, speech data and image data into the heterogeneous data encoder respectively for feature extraction to obtain a text feature vector, a speech feature vector and an image feature vector of the prompt task; Based on the attention mechanism, weights are assigned to the text feature vector, speech feature vector, and image feature vector of the prompt task, and based on the assigned weights, the text feature vector, speech feature vector, and image feature vector are cross-modally fused to obtain a cross-modally fused feature vector. The cross-modal fused feature vector is input into a pre-trained large language model to generate an initial prompt word set that matches the prompt task.

3. The method for generating prompt words for a large model according to claim 1, characterized in that: The step of constructing a structured knowledge graph and an unstructured database associated with the prompt task, and injecting domain knowledge into the initial prompt word set based on the structured knowledge graph and the unstructured database to generate a domain-enhanced candidate prompt word set includes: Constructing a structured knowledge graph based on the prompt task, matching the keywords of the initial prompt word set with entity nodes in the knowledge graph, and obtaining target entities associated with the keywords in the initial prompt word set and association paths between target entities; Performing a text paragraph search on the unstructured database of the prompt task, and calculating the semantic similarity between the text paragraphs in the unstructured database and the initial prompt word set, and screening out text paragraphs whose semantic similarity is greater than a preset semantic similarity threshold; The association paths between the target entities are fused with the screened text paragraphs to generate a domain-enhanced candidate prompt word set.

4. The method for generating prompt words for a large model according to claim 3, characterized in that: The step of fusing the association paths between the target entities with the screened text paragraphs to generate a domain-enhanced candidate prompt word set includes: Perform graph attention encoding on the association paths between target entities to generate structured knowledge feature vectors; Semantically encode the selected text paragraphs to generate unstructured knowledge feature vectors; Based on the attention weight, the structured knowledge feature vector and the unstructured knowledge feature vector are weightedly spliced, and the weighted spliced ​​feature vector is input into the prompt word generator to generate a domain-enhanced candidate prompt word set.

5. The method for generating prompt words for a large model according to claim 1, characterized in that: The constructing of the intelligent agent framework, inputting the candidate prompt word set into the intelligent agent framework for semantic feature extraction, and generating a target prompt word set based on the extracted semantic features, includes: Inputting the candidate prompt word set into the intelligent agent framework to extract the user intention, knowledge domain and semantic constraints of the prompt task; A prompt word template generator is constructed based on the extracted user intention, knowledge domain and semantic constraints of the prompt task, and the target prompt word is generated based on the constructed prompt word model generator.

6. The method for generating prompt words for a large model according to claim 1, characterized in that: After the constructed model generator generates the target prompt word, the method further includes: Obtaining explicit feedback data and implicit feedback data of the user to the target prompt word, and extracting a feedback feature vector based on the explicit feedback data and implicit feedback data of the target prompt word; The prompt word template generator is dynamically adjusted based on the feedback feature vector of the extracted target prompt word, and the target prompt word is updated based on the adjusted prompt word template generator.

7. The method for generating prompt words for a large model according to claim 1, characterized in that: The dynamically adjusting the prompt word template generator based on the extracted feedback feature vector of the target prompt word, and updating the target prompt word based on the adjusted prompt word template generator, includes: Calculate the dynamic weight of each feedback feature vector of the target prompt word based on the feature distribution of the extracted feedback feature vector of the target prompt word; Based on the calculated dynamic weights of the feedback feature vectors, the keywords and sentence structures in the prompt word template generator are dynamically adjusted, and the target prompt word is updated based on the adjusted prompt word template generator.

8. A large-scale model prompt word generation device, characterized in that: include: a processing module, configured to obtain multimodal data corresponding to the prompt task and perform data preprocessing on the multimodal data, wherein the multimodal data includes at least text data, voice data, and image data; An extraction module, configured to input the preprocessed multimodal data into a pre-trained heterogeneous data encoder for cross-modal feature extraction, and generate an initial prompt word set based on the extracted cross-modal features; A construction module is used to construct a structured knowledge graph and an unstructured database associated with the prompt task, and inject domain knowledge into the initial prompt word set based on the structured knowledge graph and the unstructured database to generate a domain-enhanced candidate prompt word set; The generation module is used to construct an intelligent agent framework, input the candidate prompt word set into the intelligent agent framework to extract semantic features, and generate a target prompt word set based on the extracted semantic features.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method for generating prompt words for a large model as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method for generating prompt words of the large model according to any one of claims 1 to 7 are implemented.

Images