Prompt word generation method and device, question and answer method and device, medium and electronic equipment

Abstract

The invention provides a cue word generation method and device, a question and answer method and device, a medium and electronic equipment. The cue word generation method comprises the steps of performing first semantic retrieval and first keyword retrieval on query information based on an analysis result library, a DDL library, an SQL-Query pair library and a noun interpretation library to obtain a first semantic retrieval result and a first keyword retrieval result; generating a first prompt word based on the first semantic retrieval result and the first keyword retrieval result; performing second semantic retrieval and second keyword retrieval on the query information based on the tool library and the prompt library associated with the cloud forensics to obtain a second semantic retrieval result and a second keyword retrieval result; and generating a second cue word based on the second keyword retrieval result and the second semantic retrieval result. According to the cue word generation method, the cue word accuracy can be improved, so that the processing capability of a large language model on intelligent questions and answers can be improved.

Description

Technical Field

[0001] The present application belongs to the technical field of artificial intelligence, and relates to a prompt word generation method, and in particular to a prompt word generation method, a question-answering method, a device, a medium, and an electronic device. Background Art

[0002] As an important part of the field of artificial intelligence, intelligent question-answering systems have been widely used in many fields such as customer service, online education, medical consultation, and electronic evidence collection. With the development of technology, users have higher and higher expectations for question-answering systems. They not only require fast and accurate answers, but also expect the system to understand the context, conduct multiple rounds of dialogue, and provide personalized services. Currently, intelligent question-answering systems on the market are mainly based on technologies such as rule matching, keyword retrieval, and machine learning. These systems usually use a single question-answering model to provide users with answers by retrieving relevant information from the database.

[0003] Because the current intelligent answers do not generate accurate prompt words when answering questions about software analysis results or cloud forensics, the large language model has low processing capabilities for intelligent question answering. Therefore, the current prompt word generation method has the problem of low accuracy of generated prompt words. Summary of the invention

[0004] The purpose of the present application is to provide a prompt word generation method, a question-answering method, a device, a medium and an electronic device, which are used to solve the problem of low accuracy of the prompt words generated by the current prompt word generation method.

[0005] In a first aspect, the present application provides a prompt word generation method based on a large language model, the prompt word generation method comprising: obtaining query information of a user; processing the query information to obtain a question type of the query information, the question type being a question type about a software analysis result or a question type about cloud forensics; when the question type is the question type about the software analysis result, based on an analysis result library, a DDL library, an SQL-Query pair library, and a term interpretation library associated with the software, performing a first semantic search and a first keyword search on the query information to obtain a first semantic search result and a first keyword search result; generating a first prompt word based on the first semantic search result and the first keyword search result, the analysis result library, the DDL library, and the SQL-Query pair library are all associated with the software analysis result, and the SQL-Query pair is a pair of preset query information and its associated SQL query statement; when the question type is the question type about cloud forensics, performing a second semantic search and a second keyword search on the query information based on a tool library and a prompt library associated with cloud forensics to obtain a second semantic search result and a second keyword search result; generating a second prompt word based on the second keyword search result and the second semantic search result.

[0006] In the generation method, corresponding prompt words, namely, first prompt words and second prompt words, are generated for different question types. By performing keyword retrieval and semantic retrieval in the analysis result library, DDL library, SQL-Query pair library and term explanation library, and generating the first prompt word according to the retrieval results, prompt words with high accuracy for question types related to software analysis results can be generated. By performing keyword retrieval and semantic retrieval in the tool library and prompt library associated with cloud forensics, and generating the second prompt word according to the retrieval results, prompt words with high accuracy for question types related to cloud forensics can be generated, thereby improving the processing capability of the large language model for intelligent question answering.

[0007] In one embodiment of the present application, a method for processing the query information to obtain the question type of the query information includes: rewriting the query information through the Qianwen model to obtain rewritten query information, wherein the rewritten query information includes contextual query information; and classifying the query information through the Qianwen model to obtain the question type of the query information.

[0008] In one embodiment of the present application, based on the analysis result library, the DDL library, the SQL-Query pair library and the term interpretation library associated with the software, the query information is subjected to a first semantic search and a first keyword search to obtain a first semantic search result and a first keyword search result. The implementation method includes: performing vector conversion processing on the query information, the analysis result library, the DDL library, the SQL-Query pair library and the term interpretation library to obtain a first query information vector, the analysis result vector library, the DDL vector library, the SQL-Query pair vector library and the term interpretation vector library; performing a first semantic search and a first keyword search on the first query information vector in the analysis result vector library, the DDL vector library, the SQL-Query pair vector library and the term interpretation vector library, respectively, to obtain the first semantic search result and the first keyword search result.

[0009] In one embodiment of the present application, a method for generating a first prompt word based on the first semantic search result and the first keyword search result includes: processing the first semantic search result and the first keyword search result to obtain a mixed deduplicated search result; reordering the mixed deduplicated search result through a semantic rearrangement model to obtain a reordered search result; filtering the reordered search result through a large language model to obtain a filtered search result, wherein the filtered search result includes a filtered DDL, a filtered SQL-Query pair, and a filtered term explanation; processing a prompt word spliced ​​by the filtered DDL, the filtered SQL-Query pair, and the filtered term explanation through the large language model to obtain a target SQL query statement; executing the target SQL query statement through the large language model to obtain a target SQL execution result; splicing the filtered search result and the target SQL execution result to obtain the first prompt word.

[0010] In one embodiment of the present application, based on the tool library and prompt library associated with the cloud forensics, a second semantic search and a second keyword search are performed on the query information to obtain a second semantic search result and a second keyword search result. The implementation method includes: performing vector conversion processing on the query information, the tool library and the prompt library to obtain a second query information vector, a tool vector library and a prompt vector library; performing a second semantic search and a second keyword search on the second query information vector in the tool vector library and the prompt vector library, respectively, to obtain the second semantic search result and the second keyword search result.

[0011] In one embodiment of the present application, an implementation method for generating a second prompt word based on the second keyword retrieval result and the second semantic retrieval result includes: filtering the prompt vectors in the second keyword retrieval result and the second semantic retrieval result based on the cosine similarity between the prompt vectors in the second keyword retrieval result and the second semantic retrieval result to obtain a filtered prompt vector; filtering the tool vectors in the second keyword retrieval result and the second semantic retrieval result to obtain a filtered tool vector; generating the second prompt word based on the filtered prompt vector and the filtered tool vector.

[0012] In a second aspect, the present application provides an intelligent question-answering method, comprising: obtaining the first prompt word or the second prompt word according to the prompt word generation method of any one of the first aspects; when the question type is the question type about cloud forensics, performing parameter verification on the second prompt word through the large language model to obtain a parameter verification result, and the parameter verification result is missing tool parameters or no missing tool parameters; if the parameter verification result is the missing tool parameters, asking the user about the missing tool parameters through the large language model, otherwise answering the query information based on the second prompt word through the large language model; when the question type is the question type about the software analysis result, answering the query information based on the first prompt word through the large language model.

[0013] In a third aspect, the present application provides a prompt word generation device, the prompt word generation device comprising: a query information acquisition module, used to obtain the query information of the user; a question type acquisition module, used to process the query information to obtain the question type of the query information, the question type being a question type about the software analysis result or a question type about cloud forensics; a first prompt word generation module, used to perform a first semantic search and a first keyword search on the query information based on an analysis result library, a DDL library, an SQL-Query pair library, and a term interpretation library associated with the software when the question type is the question type about the software analysis result, so as to obtain a first semantic search result and a first keyword search result. results; generating a first prompt word based on the first semantic retrieval result and the first keyword retrieval result, the analysis result library, the DDL library, and the SQL-Query pair library are all associated with the software analysis result, and the SQL-Query pair is a pair of preset query information and its associated SQL query statement; a second prompt word generation module is used to perform a second semantic search and a second keyword search on the query information based on the tool library and prompt library associated with cloud forensics when the problem type is the problem type about cloud forensics, so as to obtain a second semantic retrieval result and a second keyword retrieval result; generating a second prompt word based on the second keyword retrieval result and the second semantic retrieval result.

[0014] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the generation method and / or the intelligent question-answering method described in any one of the first aspects of the present application.

[0015] In a fifth aspect, the present application provides an electronic device, comprising: a memory storing a computer program; a processor communicatively connected to the memory, and executing the generation method and / or the intelligent question and answer method described in any one of the first aspects of the present application when the computer program is called.

[0016] As described above, the prompt word generation method, question-answering method, device, medium, and electronic device of the present application have the following beneficial effects:

[0017] In the generation method, corresponding prompt words, namely, first prompt words and second prompt words, are generated for different question types. By performing keyword retrieval and semantic retrieval in the analysis result library, DDL library, SQL-Query pair library and term explanation library, and generating the first prompt word according to the retrieval results, prompt words with high accuracy for question types related to software analysis results can be generated. By performing keyword retrieval and semantic retrieval in the tool library and prompt library associated with cloud forensics, and generating the second prompt word according to the retrieval results, prompt words with high accuracy for question types related to cloud forensics can be generated, thereby improving the processing capability of the large language model for intelligent question answering. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 Shown is a schematic diagram of the hardware structure for running the prompt word generation method.

[0019] Figure 2 Shown is a flowchart of the prompt word generation method described in an embodiment of the present application.

[0020] Figure 3 Shown is a flowchart of an implementation method for processing the query information to obtain the question type of the query information as described in an embodiment of the present application.

[0021] Figure 4 Shown is a flowchart of an implementation method of the present application embodiment, which performs a first semantic search and a first keyword search on the query information based on an analysis result library, a DDL library, an SQL-Query pair library, and a term explanation library associated with the software to obtain a first semantic search result and a first keyword search result.

[0022] Figure 5 Shown is a flowchart of an implementation method of generating a first prompt word based on the first semantic search result and the first keyword search result in an embodiment of the present application.

[0023] Figure 6 Shown is a flowchart of an implementation method of the present application embodiment, which performs a second semantic search and a second keyword search on the query information based on the tool library and prompt library associated with the cloud forensics to obtain a second semantic search result and a second keyword search result.

[0024] Figure 7 Shown is a flowchart of an implementation method for generating a second prompt word based on the second keyword search result and the second semantic search result in an embodiment of the present application.

[0025] Figure 8 Shown is a flow chart of the intelligent question-answering method described in an embodiment of the present application.

[0026] Fig. 9Shown is a flow chart of the intelligent question-answering method described in an embodiment of the present application.

[0027] Fig.10 Shown is a schematic diagram of the structure of the prompt word generating device described in an embodiment of the present application.

[0028] Component number description

[0029] 10 Electronic devices

[0030] 110 Memory

[0031] 120 processors

[0032] 1210 CPU

[0033] 1220 Neural Network Processor

[0034] 12210 Neural Network Implementation Engine

[0035] 12220 Dedicated Hardware Circuit

[0036] 122210 Matrix calculation unit

[0037] 122220 Vector Computing Unit

[0038] 1000 Prompt word generation device

[0039] 1010 Query information acquisition module

[0040] 1020 Question type acquisition module

[0041] 1030 First prompt word generation module

[0042] 1040 Second prompt word generation module

[0043] Steps S11-S14

[0044] Steps S21-S22

[0045] S31-S32 Steps

[0046] Steps S41-S46

[0047] Steps S51-S52

[0048] Steps S61-S63

[0049] Steps S71-S73 DETAILED DESCRIPTION

[0050] The following describes the embodiments of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0051] It should be noted that the illustrations provided in the following embodiments are only used to illustrate the basic concept of the present application in a schematic manner, and therefore the illustrations only show components related to the present application rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0052] The technical solutions in the embodiments of the present application are described in detail below in conjunction with the drawings in the embodiments of the present application.

[0053] The prompt word generation method and / or intelligent question answering method provided in the embodiments of the present application can be executed in a computer electronic device. Figure 1 For example, Figure 1 The hardware structure block diagram of the electronic device for the prompt word generation and / or intelligent question answering method. The electronic device 10 includes a memory 110 and a processor 120. The processor 120 can be a central processing unit 1210 or a dedicated neural network processor 1220. The neural network processor 1220 includes a neural network implementation engine 12210 and a dedicated hardware circuit 12220. The dedicated hardware circuit 12220 includes a matrix calculation unit 122210 and a vector calculation unit 122220.

[0054] Optionally, the neural network processor 1220 is a processor that uses a dedicated hardware circuit 12220 to perform neural network calculations, and the dedicated hardware circuit 12220 is an integrated circuit for performing neural network calculations and includes a matrix calculation unit 122210 and a vector calculation unit 122220 for hardware execution of vector-matrix multiplication.

[0055] Optionally, the neural network implementation engine 12210 is used to generate instructions for execution by the dedicated hardware circuit 12220. When the instructions are executed by the dedicated hardware circuit 12220, the dedicated hardware circuit 12220 performs operations specified by the neural network to generate a neural network output from the received neural network input.

[0056] like Figure 2 As shown, this embodiment provides a prompt word generation method, and the prompt word generation method includes:

[0057] S11, obtaining the user's query information.

[0058] Optionally, the user's query information may be user input information to a large language model.

[0059] S12: Process the query information to obtain a question type of the query information, where the question type is a question type regarding software analysis results or a question type regarding cloud forensics.

[0060] Optionally, the problem type regarding software analysis results may refer to the problem type regarding FireEye evidence analysis software analysis results. The problem type regarding cloud forensics may refer to the problem type requiring automatic invocation of a cloud forensics API.

[0061] Optionally, the method for implementing processing the query information to obtain the question type of the query information includes: processing the query information through a Qianwen model to obtain the question type of the query information.

[0062] S13, when the question type is the question type about the software analysis result, based on the analysis result library, the DDL library, the SQL-Query pair library and the term explanation library associated with the software, a first semantic search and a first keyword search are performed on the query information to obtain a first semantic search result and a first keyword search result; a first prompt word is generated based on the first semantic search result and the first keyword search result, the analysis result library, the DDL library, and the SQL-Query pair library are all associated with the software analysis result, and the SQL-Query pair is a pair of preset query information and its associated SQL query statement.

[0063] Optionally, the analysis result library associated with the software analysis result may refer to a natural language text spliced ​​into the software analysis result according to certain rules, the DDL library may include the DDL (Data Definition Language) of all local SQLite tables of the software analysis result, the SQL-Query (Structured Query Language-Query) pair library may refer to the SQL-Query pair generated according to the table creation statement of the SQLite table and some data samples, the SQL-Query pair library may refer to a database storing SQL-Query pairs, the term interpretation library may refer to the interpretation of some proprietary names of the software, the term interpretation library may be a file, i.e., Document, the software may refer to the Fire Eye Evidence Analysis Software, the some data samples may refer to some data in the data table associated with the table creation statement of the SQLite table, the some data samples may be flexibly set according to actual conditions, and this embodiment does not explicitly limit this. SQLite, i.e., SQLite Database, is a lightweight relational database management system.

[0064] Optionally, the SQL-Query pair can be generated by LLM (Large Language Model) according to the table creation statement of the SQLite table and some data samples, and SQL verification is performed to retain the SQL-Query pair of correct SQL. The preset query information can be flexibly set according to actual conditions, and this embodiment will not be repeated here.

[0065] Optionally, the sample of the first prompt word can be expressed as:

[0066] Prompt:[{'role':'system','content':"You are a senior SQLite expert and are very good at generating SQL queries to answer user questions. Your answer should be based on the following information.\n`DDL` contains the tables needed to convert the question into SQL. The comment section of the first line of each DDL statement is a description of the table.\n`Original data` contains the table name of the table that may be related and the content of each field in the table. Each row is composed of the table name and the field name and content in the table concatenated by `-`.\n`EXAMPLES` contains historical answer examples that are highly relevant to the user's question.\n\n====DDL:\n\nBasic information - device information table DDL: CREATETABLE basic_information_device--Basic information-Device information\n(\nSerial number TEXT,\n\tKey name TEXT,\n\tName TEXT,\n\tContent TEXT,\n\tSource file TEXT,\n\tDeleted TEXT\n)\n\n\n====Original data:\n\ntable_name:basic_information_deviceBasic information-Device informationSerial number: 11,Key name: IMEI2,Name: IMEI2,Content: 353142760034010\n\ntable_name:basic_information_deviceBasic information-Device informationSerial number: 17,Key name: AppleID,Name: Apple ID login history, content: local; 1744938058@qq.com\n\ntable_name: basic_information_device Basic information - device information sequence number: 12, key name: MEID, name: MEID, content: 35314276013815\n\ntable_name: basic_information_device Basic information - device information sequence number: 24, key name: AutoLock, name: automatic lock, content: 30 seconds\n\ntable e_name:basic_information_deviceBasic information-Device informationSerial number: 25,Key name:RequirePasscode,Name:Face ID and Password-Require Password,Content:Immediately\n\nPlease remember:\n1. When generating SQL, you cannot use DDL, raw data, tables and fields that do not appear in EXAMPLES.\n2. The generated SQL statement should conform to SQLite syntax and be executable because these tables and fields really exist.\n3. Do not answer any additional content except the generated SQL statement.\n\n====EXAMPLES:\n\nINPUT:\nPlease query the time, longitude (GPS), and latitude (GPS) information of all device location records from the 'Basic Information of Mobile Phone' application. \nOUTPUT:SELECT*FROMbasic_information_device_location_record WHERE source application = 'Basic Information of Mobile Phone'\n\nINPUT:\nPlease count the total number of devices in the Basic Information - Bluetooth - Paired Devices table. \nOUTPUT:SELECT COUNT(*)FROM basic_information_bluetooth_paired_devices\n\nINPUT:\nQuery the model and name information of all devices in the basic_information_device table? \nOUTPUT:SELECT nameFROM basic_information_device WHERE key name = 'ProductName'\n\nINPUT:\nWhat are the detailed information of the Basic Information - Bluetooth - Paired Device No. 1 that I need to view? \nOUTPUT:SELECT*FROM basic_information_bluetooth_paired_devices WHERE serial number = '1'\n\nINPUT:\nI want to view the specific content of the basic information - knowledgeC - Bluetooth usage records of serial numbers 1 and 3. \nOUTPUT:SELECT*FROM basic_information_knowledgec_bluetooth_usage WHERE `serial number` IN('1','3')\n\n"},{'role':'user','content':'INPUT:\nWhat is the basic information of the sample mobile phone device?\nOUTPUT:'}].

[0067] S14, when the question type is the question type about cloud forensics, based on the tool library and prompt library associated with cloud forensics, perform a second semantic search and a second keyword search on the query information to obtain a second semantic search result and a second keyword search result; generate a second prompt word based on the second keyword search result and the second semantic search result.

[0068] Optionally, based on the tool library and prompt library associated with the cloud forensics, the method for implementing the second semantic search and the second keyword search on the query information includes: performing a second semantic search and a second keyword search on the query information in the tool library and the prompt library, respectively, to obtain the second semantic search result and the second keyword search result. Performing semantic search and keyword search in the prompt library can help LLM decompose complex problems, because the call of some APIs needs to rely on the return result of another API. Performing semantic search and keyword search in the tool library can find the tools most relevant to the query information.

[0069] Optionally, the tool library may refer to a description library of tools, the tool may refer to a tool with a list of experimental specifications required to deploy a cloud lab, the prompt library may refer to a pre-prepared prompt library, and the prompts are used to handle problems that need to be decomposed, for example: to deploy a cloud lab, you need to first obtain the ID (identifier), region (region), and zone (partition) parameters of each instance through a tool that obtains a list of instance specifications, and then ask the user which instance to use for deployment. Through integrated prompt retrieval, the general LLM can quickly acquire the ability to decompose complex problems in vertical domain scenarios, and can accurately decompose complex problems into multiple sub-problems according to given prompts, significantly improving the system's processing capabilities, and in this way, the complex problem decomposition capability of the LLM can be migrated to any vertical domain.

[0070] According to the above description, the prompt word generation method includes: obtaining the query information of the user; processing the query information to obtain the question type of the query information, the question type is a question type about the software analysis result or a question type about cloud forensics; when the question type is the question type about the software analysis result, based on the analysis result library, the DDL library, the SQL-Query pair library and the software-associated term interpretation library, the query information is subjected to a first semantic search and a first keyword search to obtain a first semantic search result and a first keyword search result; based on the first semantic search result and the first keyword search result, a first prompt word is generated, the analysis result library, the DDL library, and the SQL-Query pair library are all associated with the software analysis result, and the SQL-Query pair is a pair of preset query information and its associated SQL query statement; when the question type is the question type about cloud forensics, based on the tool library and prompt library associated with the cloud forensics, the query information is subjected to a second semantic search and a second keyword search to obtain a second semantic search result and a second keyword search result; based on the second keyword search result and the second semantic search result, a second prompt word is generated.

[0071] In the generation method, corresponding prompt words, namely, first prompt words and second prompt words, are generated for different question types. By performing keyword retrieval and semantic retrieval in the analysis result library, DDL library, SQL-Query pair library and term explanation library, and generating the first prompt word according to the retrieval results, prompt words with high accuracy for question types related to software analysis results can be generated. By performing keyword retrieval and semantic retrieval in the tool library and prompt library associated with cloud forensics, and generating the second prompt word according to the retrieval results, prompt words with high accuracy for question types related to cloud forensics can be generated, thereby improving the processing capability of the large language model for intelligent question answering.

[0072] like Figure 3 As shown, this embodiment provides a method for processing the query information to obtain the question type of the query information, including:

[0073] S21, rewriting the query information through the Qianwen model to obtain rewritten query information, where the rewritten query information includes contextual query information.

[0074] Optionally, the Qianwen model may be a Qwen2-72B-Instruct-GPTQ-Int4 model.

[0075] Optionally, the query information of the context may refer to parameters provided by the user in multiple steps, and the rewritten query information includes the parameters provided by the user in multiple steps. The specific implementation process of the rewriting process may be to use the Qwen2-72B-Instruct-GPTQ-Int4 model, with role-playing and fewshot prompts as the input of LLM, and use the output of LLM as the rewritten query information, i.e., the rewritten Query, to replace the original query information of the user, i.e., the original Query. For example, it may be: self.vanna.vn.user_message(

[0076] f"""

[0077] Please optimize the user's current questions based on the user's historical questions.

[0078] Examples:

[0079] History Query:

[0080] Please check what Zhang San and Li Si mainly talked about?

[0081] Current Query:

[0082] Please guess their relationship.

[0083] Output:

[0084] Please judge the relationship between Zhang San and Li Si based on their chat records

[0085] History Query:

[0086] What is Zhang San’s QQ account?

[0087] Current Query:

[0088] What’s his WeChat ID?

[0089] Output:

[0090] What is Zhang San’s WeChat account?

[0091] Start

[0092] History Query:

[0093] {history_querys}

[0094] Current Query:

[0095] {current_query}

[0096] Output:

[0097] """ ) ]

[0100] llm_response=self.vanna.vn.submit_prompt(message_log)

[0101] S22, classify the query information through the Qianwen model to obtain the question type of the query information.

[0102] like Figure 4 As shown, this embodiment provides a method for performing a first semantic search and a first keyword search on the query information based on an analysis result library, a DDL library, an SQL-Query pair library, and a term explanation library associated with the software to obtain a first semantic search result and a first keyword search result, including:

[0103] S31, performing vector conversion processing on the query information, the analysis result library, the DDL library, the SQL-Query pair library and the term interpretation library to obtain a first query information vector, the analysis result vector library, the DDL vector library, the SQL-Query pair vector library and the term interpretation vector library.

[0104] S32, performing a first semantic search and a first keyword search on the first query information vector in the analysis result vector library, the DDL vector library, the SQL-Query pair vector library and the term explanation vector library respectively, to obtain the first semantic search result and the first keyword search result.

[0105] Optionally, implementing semantic retrieval and keyword retrieval in the analysis result vector library, the SQL query statement vector library, the SQL-Query pair vector library and the term explanation vector library requires word segmentation, building an index library and building a keyword retrieval system based on the Okapi BM25 algorithm (BM25 algorithm), which will not be described in detail in this embodiment.

[0106] Optionally, the first semantic search result may include: several semantic search results in the analysis result vector library, several semantic search results in the DDL vector library, several semantic search results in the SQL-Query pair vector library, and several semantic search results in the term explanation vector library, and the first keyword search result may include: several keyword search results in the analysis result vector library, several keyword search results in the DDL vector library, several keyword search results in the SQL-Query pair vector library, and several keyword search results in the term explanation vector library. For example, the Top50 semantic search results and the Top50 keyword search results may be obtained in the analysis result vector library, the DDL vector library, the SQL-Query pair vector library, and the term explanation vector library.

[0107] like Figure 5 As shown, this embodiment provides a method for generating a first prompt word based on the first semantic search result and the first keyword search result, including:

[0108] S41, processing the first semantic search result and the first keyword search result to obtain a mixed and duplicate-free search result.

[0109] Optionally, the process of mixing and deduplicating the first semantic search result and the first keyword search result is not described in detail in this embodiment.

[0110] S42, reordering the mixed deduplication retrieval results through a semantic reordering model to obtain reordered retrieval results.

[0111] Optionally, the semantic re-ranking model may be a bge-reranker-large semantic re-ranking model. The process of the bge-reranker-large semantic re-ranking model re-ranking the mixed de-duplicated search results is not described in detail in this embodiment.

[0112] S43, filtering the reordered search results through a large language model to obtain filtered search results, wherein the filtered search results include filtered DDL, filtered SQL-Query pairs and filtered term explanations.

[0113] Optionally, the filtered DDL refers to the DDL that may be used, the filtered SQL-Query pairs refer to the SQL-Query pairs that may be used, and the filtered glossaries refer to the glossaries that may be used. The filtering process can be flexibly set according to actual conditions, for example, filtering can be performed based on the cosine similarity between vectors, and the specific embodiments will not be repeated here.

[0114] S44, processing the prompt words formed by concatenating the filtered DDL, the filtered SQL-Query pairs and the filtered noun explanations through the large language model to obtain a target SQL query statement.

[0115] Optionally, the target SQL query statement may refer to an SQL query statement that can solve the user's query.

[0116] Optionally, the method for obtaining a target SQL query statement by processing the prompt words spliced ​​by the filtered DDL, the filtered SQL-Query pairs and the filtered noun explanations through the large language model may include: through role-playing, the large language model acts as a SQLite expert to process the prompt words spliced ​​by the filtered DDL, the filtered SQL-Query pairs and the filtered noun explanations to obtain a target SQL query statement. The SQL query statement is used to solve the user's problem.

[0117] Optionally, the implementation method of generating a first prompt word based on the first semantic retrieval result and the first keyword retrieval result also includes: if the target SQL query statement is wrong, processing the prompt word spliced ​​by the target SQL query statement and the error information after the execution of the target SQL query statement through the large language model to obtain a corrected target SQL query statement, otherwise directly executing step S45.

[0118] S45, executing the target SQL query statement through the large language model to obtain a target SQL execution result.

[0119] Optionally, the target SQL query statement may be the corrected target SQL query statement.

[0120] S46: Concatenate the filtered search result and the target SQL execution result to obtain the first prompt word.

[0121] like Figure 6 As shown, this embodiment provides a method for performing a second semantic search and a second keyword search on the query information based on the tool library and prompt library associated with the cloud forensics to obtain a second semantic search result and a second keyword search result, including:

[0122] S51, performing vector conversion processing on the query information, the tool library and the prompt library to obtain a second query information vector, a tool vector library and a prompt vector library.

[0123] Optionally, the query information, the tool library and the prompt library are subjected to vector conversion processing to obtain a second query information vector, a tool vector library and a prompt vector library. The implementation method includes: performing vector conversion processing on the query information, the tool library and the prompt library through a bge-large-zh-v1.5 semantic vector model to obtain a second query information vector, a tool vector library and a prompt vector library.

[0124] S52, performing a second semantic search and a second keyword search on the second query information vector in the tool vector library and the prompt vector library respectively, to obtain the second semantic search result and the second keyword search result.

[0125] Optionally, the second semantic search result may include: several semantic search results in the tool vector library and several semantic search results in the prompt vector library, and the second keyword search result may include: several keyword search results in the tool vector library and several keyword search results in the prompt vector library. For example, the second semantic search result may include the most relevant top10 tool vectors retrieved from the tool vector library and the most relevant top3 prompt vectors retrieved from the prompt vector library, and the second keyword search result may include: the most relevant top10 tool vectors retrieved from the prompt vector library and the most relevant top3 prompt vectors retrieved from the prompt vector library.

[0126] like Figure 7 As shown, this embodiment provides a method for generating a second prompt word based on the second keyword search result and the second semantic search result, including:

[0127] S61: Filter the hint vectors in the second keyword search result and the second semantic search result based on the cosine similarity between the hint vectors in the second keyword search result and the second semantic search result to obtain filtered hint vectors.

[0128] Optionally, the filtering process may refer to filtering out the prompts whose cosine similarity between the prompt vectors in the second keyword search result and the second semantic search result is less than 0.2, and the filtered prompt vectors may be empty.

[0129] S62: Filter the tool vectors in the second keyword search result and the second semantic search result to obtain filtered tool vectors.

[0130] Optionally, the filtered tool vector may be empty.

[0131] S63: Generate the second prompt word based on the filtered prompt vector and the filtered tool vector.

[0132] like Figure 8 As shown, this embodiment provides an intelligent question-answering method, including:

[0133] S71: Acquire the first prompt word or the second prompt word according to the generation method.

[0134] S72, when the question type is the question type about cloud forensics, perform parameter verification on the second prompt word through the large language model to obtain a parameter verification result, and the parameter verification result is missing tool parameters or no missing tool parameters; if the parameter verification result is the missing tool parameters, ask the user about the missing tool parameters through the large language model, otherwise answer the query information based on the second prompt word through the large language model.

[0135] Optionally, when the question type is a question type about cloud forensics, the query information may be answered by a first Agent, i.e., a first intelligent agent, where the first Agent is an intelligent agent for answering query information of the question type about cloud forensics.

[0136] S73: When the question type is the question type about the software analysis result, answer the query information through the large language model based on the first prompt word.

[0137] Optionally, when the question type is a question type about software analysis results, the query information can be answered by a second Agent, that is, a second intelligent agent, where the second Agent is an intelligent agent for answering query information of the question type about software analysis results.

[0138] In one embodiment of the present application, the process of the first Agent answering the query information may include: rewriting the user's input query (query information) to ensure that each rewritten query can contain the user's clear intention, and using the rewritten query to perform tool retrieval and prompt retrieval respectively. The purpose of tool retrieval is to find the tools most relevant to the user's query so that LLM can call the cloud forensics API (Application Programming Interface) to solve the user's problem. The purpose of prompt retrieval is to help LLM decompose complex problems, because the call of some APIs needs to rely on the return result of another API. Its architecture may include: multi-round dialogue query rewriting module, tool retrieval, prompt retrieval module, tool parameter verification module, tool call module, and LLM response module.

[0139] In one embodiment of the present application, the process of the second Agent answering the query information may include: rewriting the user's input query to ensure that each rewritten query can contain the user's clear intention, and using the rewritten query to perform analysis result retrieval, SQL query statement retrieval, SQL-Query pair retrieval, and Document retrieval. Then use LLM to filter the retrieved content, filter out the most relevant content, splice it into the prompt of LLM, generate SQL, and verify the correctness of SQL. If the generated SQL is wrong, use LLM to automatically correct it according to the error message. If the generated SQL is correct, execute the SQL statement, put the execution result and the retrieved analysis result into the context of LLM, and let LLM answer the user's question according to the content of the context. Its architecture may include: multi-round dialogue query rewriting module, analysis result retrieval module, SQL query statement retrieval module, SQL-Query pair retrieval module, and Document retrieval module. Document retrieval can refer to the above-mentioned term explanation retrieval.

[0140] See also Fig. 9 , Fig. 9 A flow chart of the intelligent question-answering method is shown, and the specific embodiment will not be described here.

[0141] According to the above description, the intelligent question-answering method includes: obtaining the first prompt word or the second prompt word according to the generation method; when the question type is the question type about cloud forensics, performing parameter verification on the second prompt word through the large language model to obtain a parameter verification result, and the parameter verification result is the lack of tool parameters or the absence of tool parameters; if the parameter verification result is the lack of tool parameters, then asking the user about the missing tool parameters through the large language model, otherwise answering the query information based on the second prompt word through the large language model. When the question type is the question type about software analysis results, answering the query information based on the first prompt word through the large language model.

[0142] In the intelligent question-answering method, agents on multiple products can be integrated into a chat window through multi-agent routing, and the user operation page can be simplified, which verifies the feasibility of mutual cooperation between multiple agents in the future. The integrated query rewriting module effectively improves LLM's ability to handle complex problems, the user's unclear questions, and the ability to handle multiple rounds of questions and answers, significantly improving the accuracy of the system. The integrated prompt retrieval module enables the general LLM to quickly have the ability to decompose complex problems in vertical field scenarios, and can accurately decompose complex problems into multiple sub-problems according to given prompts, significantly improving the system's processing capabilities, and in this way, the LLM's complex problem decomposition capabilities can be migrated to any vertical field. The integrated parameter verification module, when the user's input lacks parameters, LLM will not fabricate false parameters to pass, but ask the user to supplement the missing parameters by asking questions, effectively reducing the model's illusion and improving the user's interactive experience. The integrated Text2SQL module effectively improves the accuracy of generated SQL by pre-generating SQL-Query pairs combined with retrieval, and integrates SQL verification to further ensure that the generated SQL is correct, significantly improving the accuracy of the system.

[0143] The protection scope of the prompt word generation method described in the embodiment of the present application is not limited to the execution order of the steps listed in this embodiment. All solutions implemented by adding, reducing or replacing steps in the prior art based on the principles of the present application are included in the protection scope of the present application.

[0144] like Fig.10 As shown, this embodiment provides a prompt word generating device 1000, and the prompt word generating device 1000 includes:

[0145] The query information acquisition module 1010 is used to acquire the user's query information.

[0146] The question type acquisition module 1020 is used to process the query information to obtain the question type of the query information, where the question type is a question type regarding software analysis results or a question type regarding cloud forensics.

[0147] The first prompt word generating module 1030 is used to, when the question type is the question type about the software analysis result, perform a first semantic search and a first keyword search on the query information based on the analysis result library, DDL library, SQL-Query pair library and term explanation library associated with the software analysis result to obtain a first semantic search result and a first keyword search result; generate a first prompt word based on the first semantic search result and the first keyword search result, and the SQL-Query pair is the query statement and the query information pair.

[0148] The second prompt word generation module 1040 is used to perform a second semantic search and a second keyword search on the query information when the question type is the question type about cloud forensics, based on the tool library and prompt library associated with cloud forensics, to obtain a second semantic retrieval result and a second keyword retrieval result; and generate a second prompt word based on the second keyword retrieval result and the second semantic retrieval result.

[0149] In the prompt word generating device 1000 provided in this embodiment, the query information obtaining module 1010 and Figure 2 The question type acquisition module 1020 corresponds to step S11 of the prompt word generation method shown in FIG. Figure 2 The prompt word generation method shown in FIG. 1 corresponds to step S12, wherein the first prompt word generation module 1030 is connected to Figure 2 The second prompt word generating module 1040 corresponds to step S13 of the prompt word generating method shown in FIG. Figure 2 This corresponds to step S14 of the prompt word generating method shown.

[0150] In an embodiment of the present application, "first semantic retrieval", "first keyword retrieval", "first prompt word", and "first query information" respectively represent semantic retrieval, keyword retrieval, prompt word and query information when the question type is the question type about the software analysis result, and "second semantic retrieval", "second keyword retrieval", "second prompt word" and "second query information" respectively represent semantic retrieval, keyword retrieval, prompt word and query information when the question type is the question type about cloud forensics.

[0151] In the several embodiments provided in the present application, it should be understood that the disclosed device or method can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of modules / units is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or units can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules or units, which can be electrical, mechanical or other forms.

[0152] The modules / units described as separate components may or may not be physically separated, and the components displayed as modules / units may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules / units may be selected according to actual needs to achieve the purpose of the embodiments of the present application. For example, the functional modules / units in the various embodiments of the present application may be integrated into one processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into one module / unit.

[0153] Those of ordinary skill in the art should further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0154] This embodiment provides an electronic device, the electronic device comprising a memory storing a computer program; a processor connected to the memory for communication, and executing the computer program when the computer program is called. Figure 2 The prompt word generation method shown and / or Figure 8 The intelligent question answering method shown.

[0155] The present application embodiment also provides a computer-readable storage medium. A person of ordinary skill in the art can understand that all or part of the steps in the method for implementing the above embodiment can be completed by a program to instruct the processor, and the program can be stored in a computer-readable storage medium, and the storage medium is a non-transitory medium, such as a random access memory, a read-only memory, a flash memory, a hard disk, a solid-state hard disk, a magnetic tape, a floppy disk, an optical disc, and any combination thereof. The above storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a digital video disc (DVD)), or a semiconductor medium (e.g., a solid-state hard disk (SSD)), etc.

[0156] The present application embodiment may also provide a computer program product, the computer program product including one or more computer instructions. When the computer instructions are loaded and executed on a computing device, the process or function described in the embodiment of the present application is generated in whole or in part. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer or data center.

[0157] When the computer program product is executed by a computer, the computer executes the method described in the above method embodiment. The computer program product may be a software installation package, and when the above method is required, the computer program product may be downloaded and executed on a computer.

[0158] The descriptions of the processes or structures corresponding to the above-mentioned figures have different emphases. For parts that are not described in detail in a certain process or structure, please refer to the relevant descriptions of other processes or structures.

[0159] The above embodiments are merely illustrative of the principles and effects of the present application and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.

Claims

1. A method for generating a prompt word, characterized in that: The prompt word generation method comprises: Get the user's query information; Processing the query information to obtain a question type of the query information, where the question type is a question type regarding software analysis results or a question type regarding cloud forensics; When the question type is the question type about the software analysis result, based on the analysis result library, the data definition language DDL library, the structured query language-query information SQL-Query pair library and the term explanation library associated with the software, a first semantic search and a first keyword search are performed on the query information to obtain a first semantic search result and a first keyword search result; a first prompt word is generated based on the first semantic search result and the first keyword search result, the analysis result library, the DDL library, and the SQL-Query pair library are all associated with the software analysis result, and the SQL-Query pair is a pair of preset query information and its associated SQL query statement; When the question type is the question type regarding cloud forensics, based on the tool library and prompt library associated with cloud forensics, a second semantic search and a second keyword search are performed on the query information to obtain second semantic retrieval results and second keyword retrieval results; and a second prompt word is generated based on the second keyword retrieval results and the second semantic retrieval results.

2. The method for generating prompt words according to claim 1, characterized in that: The method for processing the query information to obtain the question type of the query information includes: Rewrite the query information through the Qianwen model to obtain rewritten query information, the rewritten query information includes contextual query information, and the query information is replaced by the rewritten query information; The rewritten query information is classified and processed by the Qianwen model to obtain the question type.

3. The method for generating prompt words according to claim 2, characterized in that: Based on the analysis result library, the DDL library, the SQL-Query pair library and the term explanation library associated with the software, a first semantic search and a first keyword search are performed on the query information to obtain a first semantic search result and a first keyword search result. The implementation method includes: Performing vector conversion processing on the query information, the analysis result library, the DDL library, the SQL-Query pair library and the noun interpretation library to obtain a first query information vector, the analysis result vector library, the DDL vector library, the SQL-Query pair vector library and the noun interpretation vector library; A first semantic search and a first keyword search are performed on the first query information vector in the analysis result vector library, the DDL vector library, the SQL-Query pair vector library and the term explanation vector library respectively to obtain the first semantic search result and the first keyword search result.

4. The method for generating prompt words according to claim 1, characterized in that: The method for generating a first prompt word based on the first semantic search result and the first keyword search result includes: Processing the first semantic search result and the first keyword search result to obtain a mixed and duplicate-free search result; Reordering the mixed deduplication retrieval results through a semantic reordering model to obtain reordered retrieval results; Filtering the reordered search results by using a large language model to obtain filtered search results, wherein the filtered search results include filtered DDL, filtered SQL-Query pairs, and filtered glossaries; Processing the prompt words spliced ​​by the filtered DDL, the filtered SQL-Query pairs and the filtered noun explanations through the large language model to obtain a target SQL query statement; Execute the target SQL query statement through the large language model to obtain a target SQL execution result; The filtered search results and the target SQL execution results are concatenated to obtain the first prompt word.

5. The method for generating prompt words according to claim 1, characterized in that: Based on the tool library and prompt library associated with the cloud forensics, a method for performing a second semantic search and a second keyword search on the query information to obtain a second semantic search result and a second keyword search result includes: Performing vector conversion processing on the query information, the tool library and the prompt library to obtain a second query information vector, a tool vector library and a prompt vector library; A second semantic search and a second keyword search are performed on the second query information vector in the tool vector library and the prompt vector library respectively to obtain the second semantic search result and the second keyword search result.

6. The method for generating prompt words according to claim 1, characterized in that: The method for generating a second prompt word based on the second keyword search result and the second semantic search result includes: filtering the hint vectors in the second keyword search result and the second semantic search result based on the cosine similarity between the hint vectors in the second keyword search result and the second semantic search result to obtain a filtered hint vector; filtering the tool vectors in the second keyword search result and the second semantic search result to obtain filtered tool vectors; The second prompt word is generated based on the filtered prompt vector and the filtered tool vector.

7. An intelligent question-answering method, characterized in that: The intelligent question-answering method comprises: Acquire the first prompt word or the second prompt word according to the prompt word generation method according to any one of claims 1 to 6; When the question type is the question type about cloud forensics, performing parameter verification on the second prompt word through the large language model to obtain a parameter verification result, and the parameter verification result is missing tool parameters or not missing tool parameters; if the parameter verification result is the missing tool parameters, asking the user about the missing tool parameters through the large language model, otherwise answering the query information through the large language model based on the second prompt word; When the question type is the question type regarding the software analysis result, the query information is answered by the large language model based on the first prompt word.

8. A prompt word generating device, characterized in that: The prompt word generating device comprises: A query information acquisition module is used to obtain the user's query information; A question type acquisition module, used for processing the query information to obtain the question type of the query information, wherein the question type is a question type about software analysis results or a question type about cloud forensics; A first prompt word generating module, used for, when the question type is the question type about the software analysis result, performing a first semantic search and a first keyword search on the query information based on the analysis result library, the DDL library, the SQL-Query pair library and the term explanation library associated with the software to obtain a first semantic search result and a first keyword search result; generating a first prompt word based on the first semantic search result and the first keyword search result, wherein the analysis result library, the DDL library and the SQL-Query pair library are all associated with the software analysis result, and the SQL-Query pair is a pair of preset query information and its associated SQL query statement; A second prompt word generation module is used for, when the question type is the question type about cloud forensics, performing a second semantic search and a second keyword search on the query information based on the tool library and prompt library associated with cloud forensics to obtain a second semantic search result and a second keyword search result; and generating a second prompt word based on the second keyword search result and the second semantic search result.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the generation method described in any one of claims 1 to 6 and / or the intelligent question-answering method described in claim 7.

10. An electronic device, characterized in that: The electronic device comprises: A memory storing a computer program; A processor is communicatively connected to the memory, and executes the generation method described in any one of claims 1 to 6 and / or the intelligent question-answering method described in claim 7 when calling the computer program.

Images