Intelligent question and answer processing method and device, electronic equipment and storage medium

Abstract

The invention discloses an intelligent question and answer processing method and device, electronic equipment and a storage medium, and relates to the technical field of artificial intelligence. Metadata is constructed based on data information of data in an obtained data set; converting data contents in the data set into digital vectors; performing corresponding storage based on data contents, data information, metadata and digital vectors in the data set to obtain a plurality of data records; adopting a BM25 retriever to obtain a first candidate result related to the user query, a first correlation score of the first candidate result, a second candidate result related to the user query and a second correlation score of the second candidate result, and further determining a comprehensive score; determining a target candidate result based on the comprehensive score; on the basis of the target candidate result and the user query, constructing prompt words of an intelligent question and answer model; and outputting a query result corresponding to the user query by using the intelligent question and answer model based on the cue word. Therefore, the question and answer accuracy is improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an intelligent question-answering processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] Earth observation / remote sensing data platforms aggregate large-scale datasets from different organizations and units, covering a variety of application fields. Data retrieval and question answering are usually based on Earth observation / remote sensing data platforms.

[0003] In related technologies, the document is first sliced, dividing each long document into multiple segments. Each segment is then vector-encoded to obtain multiple vector fragments, which are stored in a vector database. User queries are vector-encoded to obtain query vectors. Simultaneously, candidate vector fragments matching the query vectors are obtained based on keyword retrieval and vector retrieval. Target vector fragments matching the query vectors are then determined based on preset rules. Finally, a large model generates the answer based on the query vectors and the target vector fragments matching them.

[0004] However, using the above methods for data retrieval and question answering has the following drawbacks: users cannot verify the authenticity of answers because there is no source; the output format is chaotic, which is not conducive to subsequent processing; and the current prompts are mostly applicable to English scenarios and are not suitable for Chinese + professional field scenarios. Summary of the Invention

[0005] In view of this, this application provides an intelligent question-answering processing method, apparatus, electronic device, and storage medium to solve the problem of low accuracy of the generated answers in existing question-answering processing methods.

[0006] The objective of this application can be achieved through the following technical solutions: The first aspect of this application is to provide an intelligent question-answering processing method, including: Retrieve data information from the dataset; Metadata is constructed based on data information; Convert the data in the dataset into numerical vectors; Multiple data records are obtained by storing the corresponding data content, data information, metadata, and numerical vectors in the dataset; The BM25 algorithm is used to construct a BM25 search engine based on data records; Based on the user query, the BM25 search engine is used to obtain the first candidate results related to the user query and the first relevance score of the first candidate results. Based on the query vector and data records corresponding to the user query, a second candidate result related to the user query and a second relevance score of the second candidate result are obtained. The composite score is determined based on the first and second correlation scores. Target candidate results are determined based on the comprehensive score; Suggestion words for building an intelligent question-answering model based on target candidate results and user queries; Based on prompts, the system uses an intelligent question-answering model to output the corresponding query results for the user's query.

[0007] In one optional embodiment, a comprehensive score is determined based on a first relevance score and a second relevance score using the following formula: ; in, This represents the overall score. Represents the weight parameters. This represents the first relevance score. This represents the second relevance score.

[0008] In one optional embodiment, after obtaining multiple data records based on the data content, data information, metadata, and numerical vector correspondence storage in the dataset, the method further includes: If the metadata in a data record includes a data identifier, the data record is deduplicated using the data identifier; If the metadata in a data record does not include a data identifier, the content identifier is determined from the data content, and the data record is deduplicated based on the content identifier. If the data content does not include a content identifier, then the data content is hashed to obtain a hash value, and the data records are deduplicated based on the hash value.

[0009] In an optional embodiment, after determining the target candidate result based on the comprehensive score, the method further includes: generating a result record based on the target candidate result, wherein the result record includes a data identifier, data time information, data summary key points, data download link, and data identifier; The prompt words used to build an intelligent question-answering model based on target candidate results and user queries include: Prompt words for building an intelligent question-answering model based on result records and user queries.

[0010] In an optional embodiment, after constructing the prompt words for the intelligent question-answering model based on the target candidate results and the user query, the method further includes: Set constraints; Based on prompts, the system uses an intelligent question-answering model to output the query results corresponding to the user's query, including: Based on constraints and prompts, an intelligent question-answering model is used to output the query results corresponding to the user's query.

[0011] In one alternative embodiment, converting the data content in the dataset into a numeric vector includes: Convert the data in the dataset into numerical vectors using a preset batch size.

[0012] In an alternative embodiment, after converting the data in the dataset into numeric vectors, the method further includes: Normalize the numerical vector.

[0013] A second aspect of this application is to provide an intelligent question-and-answer processing device, characterized in that it includes: The acquisition module is used to retrieve data information from the dataset. The first building module is used to construct metadata based on data information; The conversion module is used to convert the data content in the dataset into numerical vectors; The first determining module is used to obtain multiple data records based on the data content, data information, metadata, and corresponding storage of numerical vectors in the dataset; The second building module is used to construct a BM25 search engine based on data records using the BM25 algorithm. The second determination module is used to obtain the first candidate results related to the user query and the first relevance score of the first candidate results based on the user query and using the BM25 retrieval tool. The third determination module is used to obtain the second candidate result related to the user query and the second relevance score of the second candidate result based on the query vector and data record corresponding to the user query. The fourth determination module is used to determine the comprehensive score based on the first correlation score and the second correlation score; The fifth determination module is used to determine the target candidate results based on the comprehensive score; The third building module is used to construct prompt words for the intelligent question-answering model based on the target candidate results and user queries; The output module is used to output the query results corresponding to the user's query based on the prompt words and using an intelligent question-answering model.

[0014] A third aspect of this application is to provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the method as described in the first aspect.

[0015] A fourth aspect of this application is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the method as described in the first aspect.

[0016] Compared with existing technologies, the intelligent question-answering processing method provided in this application constructs metadata based on the data information of the acquired dataset; converts the data content in the dataset into numerical vectors; stores the corresponding data content, data information, metadata, and numerical vectors in the dataset to obtain multiple data records; uses the BM25 retrieval tool to obtain a first candidate result related to the user query, a first relevance score of the first candidate result, a second candidate result related to the user query, and a second relevance score of the second candidate result, thereby determining a comprehensive score; determines the target candidate result based on the comprehensive score; constructs prompt words for the intelligent question-answering model based on the target candidate results and the user query; and outputs the query results corresponding to the user query using the intelligent question-answering model based on the prompt words. This improves the accuracy of question answering. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating the intelligent question-answering processing method provided in this application embodiment; Figure 2 A structural block diagram of the intelligent question-answering processing device provided in the embodiments of this application; Figure 3 This is a structural block diagram of an electronic device for implementing an intelligent question-answering processing method, provided in an embodiment of this application. Detailed Implementation

[0019] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0020] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0021] It should be understood that in the embodiments of this application, "at least one" means one or more, and "more than one" means two or more. "And / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. The character " / " generally indicates that the related objects before and after it are in an "or" relationship. "Contains A, B and / or C" means containing any one, two, or three of A, B, and C.

[0022] It should be understood that in the embodiments of this application, "B corresponding to A", "B corresponding to A", "A corresponds to B" or "B corresponds to A" means that B is associated with A, and B can be determined based on A. Determining B based on A does not mean that B is determined solely based on A; B can also be determined based on A and / or other information.

[0023] To address the technical problems existing in related technologies, embodiments of this application provide an intelligent question-and-answer processing method, apparatus, electronic device, and storage medium.

[0024] The intelligent question-answering processing method provided in this application can be executed by an electronic device, which can be a terminal or a server. The terminal can be a smartphone, tablet, laptop, or other similar device. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. It is understood that this application does not limit the specific entity executing the intelligent question-answering processing method.

[0025] The technical solution of this application will be described in detail below through specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments described below are used to explain the technical solution of this application and are not intended to limit actual use.

[0026] To address the technical problems existing in related technologies, embodiments of this application provide an intelligent question-answering processing method, such as... Figure 1 As shown, Figure 1 This is a flowchart of an intelligent question-answering processing method provided in an embodiment of this application. It should be noted that the steps shown may be executed in a logical order different from that shown in the flowchart. The method may include the following steps S101 to S111.

[0027] Step S101: Obtain data information from the dataset.

[0028] In one alternative embodiment, the dataset in this application includes data from different organizations and units and covers a variety of application areas.

[0029] In one specific embodiment, the data can be Earth observation / remote sensing data.

[0030] In an optional embodiment, before obtaining the data information from the dataset, the following steps can be performed: A session token is requested from the interface of the data platform (which can be an Earth observation / remote sensing data platform) using a pre-configured key. This session token is used to verify the user's identity. Subsequently, the dataset list interface provided by the data platform can be called to send a request to the data platform to obtain the dataset list, which carries the session token. The dataset list returned by the data platform is received; the dataset list includes dataset identifiers. The dataset corresponding to each dataset identifier is obtained. Then, the data information from the corresponding dataset is retrieved.

[0031] In one optional embodiment, the data information includes at least one of the following: data identifier, data title, data keywords, data description, data time information, and data download link. In addition, the data information may include other information, which is not limited in this application.

[0032] In one specific embodiment, the data identifier is represented by id, the data title by title, the data keyword by keyword, the data description by description, the data time information by timeInfo, and the data download link by download URL (the network address used to download the resource, Download Uniform Resource Locator).

[0033] In a more specific embodiment, the data information of data r can be represented as: .

[0034] In an alternative embodiment, the data may be stored in JSON (JavaScript Object Notation) format for subsequent machine parsing.

[0035] In another optional embodiment, the intelligent question-answering processing method provided in this application further includes the following steps: Retry and log abnormal responses to ensure a minimum usable dataset.

[0036] In this embodiment, considering that data acquisition anomalies may occur during the process of obtaining data information from the dataset, retrying and recording abnormal responses will ensure a minimum usable dataset.

[0037] In one specific embodiment, retrying and logging abnormal responses to ensure a minimum available dataset specifically includes the following steps: If retrieving data from the dataset fails, the system will retry a preset number of times. If the problem persists after the preset number of retries, the exception from the dataset list interface, the error code, and the time of the exception will be recorded for future troubleshooting. Furthermore, if a field is missing, other fields can be retained to generate "incomplete but usable" data.

[0038] By retries and logging of abnormal responses, the minimum usable dataset is ensured to improve system availability, automatically recovering from temporary failures without the user's awareness. Even with partial failures, "incomplete but useful" results can be obtained. Logging anomalies ensures problem traceability.

[0039] Step S102: Construct metadata based on data information.

[0040] It should be noted that metadata refers to information about data.

[0041] In this embodiment, metadata is constructed to ensure the traceability of subsequent retrieval and citation.

[0042] In one alternative embodiment, the metadata includes at least one of the following: data identifier, data download link, data dataset name, and data source.

[0043] In one specific embodiment, metadata can be represented by the following expression: .in, (Application Programming Interface index) indicates the name of the dataset. Indicates the source of the data.

[0044] Step S103: Convert the data content in the dataset into a number vector.

[0045] In one alternative embodiment, converting the data content in the dataset into a numeric vector includes: Convert the data in the dataset into numerical vectors using a preset batch size.

[0046] In one alternative embodiment, the data content can be in the form of documents and web pages.

[0047] In one specific embodiment, a vector transformation model is loaded, and the data content in the dataset is converted into digital vectors using a preset batch size.

[0048] In one specific embodiment, the vector transformation model can be the BAAI / bge-large-zh model, the bge-m3 model, or the gte-large-zh model, or other vector transformation models; this application does not limit this to any particular model. It should be noted that the BAAI / bge-large-zh model, the bge-m3 model, and the gte-large-zh model are all common models used for natural language processing tasks, and this application will not elaborate on them further.

[0049] In one specific embodiment, the batch size is represented by BATCH_SIZE, for example, BATCH_SIZE=64, BATCH_SIZE=32, and BATCH_SIZE=128, etc.

[0050] In another specific embodiment, the model is migrated to the GPU (Graphics Processing Unit) when the GPU is detected to accelerate batch encoding.

[0051] In a more specific embodiment, the document is cut into multiple segments, and then the vector transformation model processes the preset batch size segments simultaneously (instead of processing them one by one), efficiently converting them into semantic vectors for subsequent retrieval or matching.

[0052] In another alternative embodiment, after converting the data content in the dataset into a numeric vector, the method further includes: Normalize the numerical vector.

[0053] In one specific embodiment, the numerical vector is L2 normalized.

[0054] In this embodiment, L2 normalization is performed on the numerical vectors to improve the stability and efficiency of similarity.

[0055] Step S104: Based on the data content, data information, metadata, and numerical vector correspondence storage in the dataset, multiple data records are obtained.

[0056] In one alternative embodiment, a vector database is established. This vector database can be a lightweight and open-source vector database, such as the Chroma vector database, FAISS (Facebook AI Similarity Searh), Milvus, Open Search k-NN (Open Search k-Nearest Neighbors), and Vespa k-NN (Vespa k-Nearest Neighbors). The aforementioned vector databases support index structures such as HNSW (Hierarchical Navigable Small World), which are not limited in this application.

[0057] In one specific implementation, the Chroma vector database's Persistent Client is used to write the data directly to the local disk, ensuring that the data is not lost even if the program is closed or the machine is restarted. For example, a directory A is specified, and the data is stored in that directory. Further, a collection for storing Earth observation data is created in that directory; for example, this collection could be named earth_observation_data.

[0058] Data identifiers can be expressed using a stable prefix and an auto-incrementing index. The prefix is ​​a defined string of characters placed at the beginning of the identifier, and "stable" means that the same rule is used for each identifier generation. For example, an identifier can be expressed as: api_#, where api indicates that the data was obtained through an API, and _# is a separator used to emphasize the subsequent number. The auto-incrementing index ensures the uniqueness of the identifier and avoids duplicate identifiers.

[0059] Through the above operations, a mapping structure of "text-vector-metadata" is formed in the vector database, supporting robust deduplication and source citation based on unique identifiers.

[0060] In another optional embodiment, after the batch data writing is completed, a set count and time consumption can be displayed. The set count is used to confirm whether the data has been completely written, and the time consumption indicates the duration of the entire data writing process. This achieves a monitorable and maintainable engineered data entry process.

[0061] In another optional embodiment, the intelligent question-answering processing method provided in this application further includes the following steps: if the metadata in the data record includes a data identifier, the data record is deduplicated using the data identifier; if the metadata in the data record does not include a data identifier, a content identifier is determined from the data content, and the data record is deduplicated based on the content identifier; if the data content does not include a content identifier, the data content is hashed to obtain a hash value, and the data record is deduplicated based on the hash value.

[0062] In this embodiment, by deduplicating data records, duplicate or highly similar candidate results are avoided, thereby improving retrieval quality and user experience; reducing the space occupied by invalid data, lowering index size and storage costs; and accelerating retrieval speed and reducing latency.

[0063] Step S105: Use the BM25 algorithm to construct a BM25 search engine based on the data records.

[0064] It should be noted that the BM (Best Matching) 25 retrieval tool is a probabilistic retrieval model used to evaluate the relevance of search terms to documents.

[0065] In one alternative embodiment, the BM25 search engine is constructed based on data content, data title, data description, and data keywords.

[0066] In one specific embodiment, the BM25 search engine is constructed based on data content, data title, data description, and data keywords, specifically including the following steps: The data title, data description, and data keywords are concatenated into a structured string and assigned to the page_content (text content) field. It should be noted that page_content is a string field and serves as input to the BM25 search engine.

[0067] In this embodiment, only the data title, data description, and data keywords are written to the `page_content` field, while other redundant / noisy fields are not included, reducing index size and term noise. Using structured formats such as JSON as data content, rather than simple string concatenation, ensures stable word order and field boundaries, facilitating score repeatability during repeated index construction. Metadata is stored separately for data identifiers and download links, rather than being mixed into the inverted text, reducing the expansion of meaningless terms.

[0068] Step S106: Based on the user query, use the BM25 search engine to obtain the first candidate result related to the user query and the first relevance score of the first candidate result.

[0069] In one optional embodiment, based on the user query, the BM25 search engine is used to obtain a first candidate result related to the user query and a first relevance score of the first candidate result, specifically including the following steps: Based on the user query, the BM25 search engine is used to obtain the top N candidate results that are relevant to the user query. The first relevance score of each candidate result is determined based on the similarity between the user query and the top N candidate results. This first relevance score can be used to... express.

[0070] It should be noted that the first candidate result is any data record among the above multiple data records.

[0071] In one alternative embodiment, the similarity can be cosine similarity, inner product, or Euclidean distance.

[0072] Step S107: Based on the query vector and data records corresponding to the user query, obtain the second candidate result related to the user query and the second relevance score of the second candidate result.

[0073] In one optional embodiment, based on the query vector and data records corresponding to the user query, a second candidate result related to the user query and a second relevance score of the second candidate result are obtained, specifically including the following steps: The user query is transformed into a query vector using a sentence embedding model; the top N second candidate results most similar to the query vector are found in the vector database. The second relevance score of the second candidate results is determined based on the similarity between the second candidate results and the query vector.

[0074] In a more specific embodiment, the Sentence Embedding model can be at least one of the following models: all-MiniLM-L6-v2, text2vec-base-chinese, and bge-small-zh-v1.5. Other models are also possible, and this application does not limit their use. It should be noted that all-MiniLM-L6-v2, text2vec-base-chinese, and bge-small-zh-v1.5 are common models for converting text into vector representations, and this application will not elaborate on them.

[0075] In one optional embodiment, based on the query vector and data records corresponding to the user query, a second candidate result related to the user query and a second relevance score of the second candidate result are obtained, including the following steps: Based on the query vector, data time information, data keywords, data description, data identifier, and data title corresponding to the user query, a second candidate result related to the user query and a second relevance score of the second candidate result are obtained.

[0076] It should be noted that the second candidate result is any data record among the above multiple data records.

[0077] In this embodiment, during vector transformation, only data time information, data keywords, data descriptions, data identifiers, and data titles are input into the vector transformation model, reducing semantic dilution and vector dimensionality noise. The data is serialized into a JSON string using a fixed-key set before vector transformation, resulting in a stable text structure that can be repeatedly constructed. Using the Chroma vector database's Persistent Client, the process does not rely on cloud-embedded / hosted vector services, satisfying privacy and controllable costs. Data identifiers, data download links, data sources, and other information are retained. Synchronous storage with digital vectors allows retrieval results to be directly output as verifiable and downloadable information without the need for a secondary database search.

[0078] Step S108: Determine the comprehensive score based on the first correlation score and the second correlation score.

[0079] In one optional embodiment, a comprehensive score is determined based on a first relevance score and a second relevance score using the following formula: (1); in, This represents the overall score. Represents the weight parameters. This represents the first relevance score. This represents the second relevance score.

[0080] It should be noted that different settings can be configured for different scenarios. For example, in short query and proper name scenarios, you can... Set it to a larger size.

[0081] In another alternative embodiment, the overall score is determined by methods such as bottom-ranking fusion or Z-score.

[0082] In another alternative embodiment, the first and second correlation scores may be normalized before determining the composite score based on the first and second correlation scores.

[0083] Step S109: Determine the target candidate results based on the comprehensive score.

[0084] In one optional embodiment, the candidate results are sorted according to their comprehensive scores, and the top M candidate results are selected as the target candidate results.

[0085] In another optional embodiment, after determining the target candidate results based on the comprehensive score, the method further includes the following step: generating a result record based on the target candidate results. The result record includes a data identifier, data time information, data summary highlights, data download link, and data identifier. It should be noted that the data summary highlights represent the most critical and core information points in the data content. Furthermore, the result record can be adapted to the actual situation, and this application does not limit its implementation.

[0086] In this embodiment, a result record is generated that includes only data identifier, data time information, data summary key points, data download link, and data identifier, so that it can be adapted to the context window of the generation stage later.

[0087] Step S110: Construct prompt words for the intelligent question-answering model based on the target candidate results and user queries.

[0088] In one alternative embodiment, the intelligent question-answering model can be an LLM (Large Language Model).

[0089] In one alternative embodiment, when generating result records based on target candidate results, prompt words for the intelligent question-answering model are constructed based on the result records and user queries.

[0090] In another alternative embodiment, after constructing the prompts for the intelligent question-answering model based on the target candidate results and the user query, the method further includes setting constraints.

[0091] In one specific embodiment, the constraint can be "No fabrication allowed, subsequent output query results must include data source and data download link, if no query results are output, then search suggestions must be provided".

[0092] Step S111: Based on the prompt words, use the intelligent question-answering model to output the query results corresponding to the user's query.

[0093] In one optional embodiment, when constraints are set, based on prompt words, an intelligent question-answering model is used to output the query results corresponding to the user's query, including: Based on constraints and prompts, an intelligent question-answering model is used to output the query results corresponding to the user's query.

[0094] In another optional embodiment, a local large-model runtime platform, such as Ollam, vLLM (vectorized Large Language Model inference engine), or TGI (Text Generation Inference engine), is used to invoke the intelligent question-answering model. The model temperature, maximum output length, and stop words of the intelligent question-answering model are set to a preset value. For example, the preset temperature can be 0.2, 0.3, or 0.5, and the preset length can be 200, 300, or 600. This application does not limit the specific content of the preset temperature, preset length, and preset stop words. This ensures that the model's answers are concise, relevant, and have fewer illusions.

[0095] In another optional embodiment, the returned query results can be in a fixed format. The fixed format can be JSON. By returning structured query results, the results possess the advantages of being "clear," "accessible," "verifiable," and "integrable." "Clear" means that users can understand relevant information about the query results, such as the data source, the data time period, and the key points of the data summary; "accessible" means that users can obtain the data content based on the data download link; "verifiable" means that the source can be directly traced; and "integrable" means that the structured data is applicable to multiple systems.

[0096] In another optional embodiment, if no query results can be provided, executable search suggestions and log identifiers can be output to facilitate background logging and optimization of search strategies.

[0097] Corresponding to the intelligent question-answering processing method provided in the embodiments of this application, the embodiments of this application also provide an intelligent question-answering processing device, such as... Figure 2 As shown, the intelligent question-answering processing device includes: Module 201 is used to obtain data information from the dataset. The first construction module 202 is used to construct metadata based on data information; Conversion module 203 is used to convert the data content in the dataset into numerical vectors; The first determining module 204 is used to obtain multiple data records based on the data content, data information, metadata and digital vector correspondence storage in the dataset; The second construction module 205 is used to construct a BM25 searcher based on data records using the BM25 algorithm; The second determination module 206 is used to obtain, based on the user query, a first candidate result related to the user query and a first relevance score of the first candidate result using the BM25 retrieval tool. The third determination module 207 is used to obtain a second candidate result related to the user query and a second relevance score of the second candidate result based on the query vector and data record corresponding to the user query. The fourth determining module 208 is used to determine the comprehensive score based on the first correlation score and the second correlation score; The fifth determination module 209 is used to determine the target candidate results based on the comprehensive score; The third construction module 210 is used to construct prompt words for the intelligent question answering model based on the target candidate results and user queries; Output module 211 is used to output the query results corresponding to the user's query based on prompt words and using an intelligent question-answering model.

[0098] Corresponding to the intelligent question-answering processing method provided in the embodiments of this application, the embodiments of this application also provide an electronic device for executing the intelligent question-answering processing method, such as... Figure 3 As shown, the electronic device includes: a processor 301; and a memory 302 for storing a program for an intelligent question-answering processing method. After the device is powered on and the program for the intelligent question-answering processing method is run by the processor, the following steps are performed: Retrieve data information from the dataset; Metadata is constructed based on data information; Convert the data in the dataset into numerical vectors; Multiple data records are obtained by storing the corresponding data content, data information, metadata, and numerical vectors in the dataset; The BM25 algorithm is used to construct a BM25 search engine based on data records; Based on the user query, the BM25 search engine is used to obtain the first candidate results related to the user query and the first relevance score of the first candidate results. Based on the query vector and data records corresponding to the user query, a second candidate result related to the user query and a second relevance score of the second candidate result are obtained. The composite score is determined based on the first and second correlation scores. Target candidate results are determined based on the comprehensive score; Suggestion words for building an intelligent question-answering model based on target candidate results and user queries; Based on prompts, the system uses an intelligent question-answering model to output the corresponding query results for the user's query.

[0099] Corresponding to the intelligent question-answering processing method provided in the embodiments of this application, the embodiments of this application also provide a computer-readable storage medium storing a program for the intelligent question-answering processing method, which is executed by a processor to perform the following steps: Retrieve data information from the dataset; Metadata is constructed based on data information; Convert the data in the dataset into numerical vectors; Multiple data records are obtained by storing the corresponding data content, data information, metadata, and numerical vectors in the dataset; The BM25 algorithm is used to construct a BM25 search engine based on data records; Based on the user query, the BM25 search engine is used to obtain the first candidate results related to the user query and the first relevance score of the first candidate results. Based on the query vector and data records corresponding to the user query, a second candidate result related to the user query and a second relevance score of the second candidate result are obtained. The composite score is determined based on the first and second correlation scores. Target candidate results are determined based on the comprehensive score; Suggestion words for building an intelligent question-answering model based on target candidate results and user queries; Based on prompts, the system uses an intelligent question-answering model to output the corresponding query results for the user's query.

[0100] Corresponding to the intelligent question-answering processing method provided in the embodiments of this application, the embodiments of this application also provide a computer program containing instructions, which, when executed by a computer, cause the computer to perform the following steps: Retrieve data information from the dataset; Metadata is constructed based on data information; Convert the data in the dataset into numerical vectors; Multiple data records are obtained by storing the corresponding data content, data information, metadata, and numerical vectors in the dataset; The BM25 algorithm is used to construct a BM25 search engine based on data records; Based on the user query, the BM25 search engine is used to obtain the first candidate results related to the user query and the first relevance score of the first candidate results. Based on the query vector and data records corresponding to the user query, a second candidate result related to the user query and a second relevance score of the second candidate result are obtained. The composite score is determined based on the first and second correlation scores. Target candidate results are determined based on the comprehensive score; Suggestion words for building an intelligent question-answering model based on target candidate results and user queries; Based on prompts, the system uses an intelligent question-answering model to output the corresponding query results for the user's query.

[0101] It should be noted that for a detailed description of the intelligent question-answering processing device, electronic device and computer-readable storage medium provided in the embodiments of this application, please refer to the relevant description of the intelligent question-answering processing method embodiments provided in the embodiments of this application, which will not be repeated here.

[0102] Although this application discloses preferred embodiments as described above, it is not intended to limit this application. Any person skilled in the art can make possible changes and modifications without departing from the spirit and scope of this application. Therefore, the scope of protection of this application should be determined by the scope defined in the claims of this application.

[0103] In a typical configuration, an electronic device includes one or more processors (Central Processing Units), input / output interfaces, network interfaces, and memory.

[0104] Memory may include non-persistent storage in computer-readable media, such as random access memory and / or non-volatile memory, like read-only memory or flash memory. Memory is an example of computer-readable media.

[0105] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable operations, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PCM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital video disc (DMCD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include non-transitory computer-readable media, such as modulated data signals and carrier waves.

[0106] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, compact disc read-only memory, optical storage, etc.) containing computer-usable program code.

[0107] Although this application discloses preferred embodiments as described above, it is not intended to limit this application. Any person skilled in the art can make possible changes and modifications without departing from the spirit and scope of this application. Therefore, the scope of protection of this application should be determined by the scope defined in the claims of this application.

Claims

1. An intelligent question-answering processing method, characterized in that, include: Retrieve data information from the dataset; Metadata is constructed based on the aforementioned data information; Convert the data content in the dataset into numerical vectors; Based on the data content, data information, metadata, and digital vectors in the dataset, multiple data records are obtained through corresponding storage. The BM25 algorithm is used to construct a BM25 search engine based on the data records; Based on the user query, the BM25 retrieval tool is used to obtain a first candidate result related to the user query and a first relevance score of the first candidate result. Based on the query vector corresponding to the user query and the data record, a second candidate result related to the user query and a second relevance score of the second candidate result are obtained. A comprehensive score is determined based on the first correlation score and the second correlation score; The target candidate results are determined based on the comprehensive score; Based on the target candidate results and the user query, construct the prompt words for the intelligent question answering model; Based on the prompt words, the intelligent question-answering model is used to output the query results corresponding to the user's query.

2. The intelligent question-answering processing method according to claim 1, characterized in that, The comprehensive score is determined based on the first correlation score and the second correlation score using the following formula: ； Among them, the The overall score is represented by the following. Represents the weight parameters, the Represents the first relevance score, the This represents the second correlation score.

3. The intelligent question-answering processing method according to claim 1, characterized in that, After obtaining multiple data records by corresponding storage of the data content, data information, metadata, and digital vectors in the dataset, the method further includes: If the metadata in the data record includes a data identifier, the data record is deduplicated using the data identifier; If the metadata in the data record does not include a data identifier, a content identifier is determined from the data content, and the data record is deduplicated based on the content identifier; If the data content does not include the content identifier, then the data content is hashed to obtain a hash value, and the data record is deduplicated based on the hash value.

4. The intelligent question-answering processing method according to claim 1, characterized in that, After determining the target candidate results based on the comprehensive score, the method further includes: generating a result record based on the target candidate results, wherein the result record includes a data identifier, data time information, data summary key points, data download link, and data identifier; The prompt words for constructing the intelligent question-answering model based on the target candidate results and the user query include: Based on the result records and the user query, prompt words are constructed to create an intelligent question-answering model.

5. The intelligent question-answering processing method according to claim 1, characterized in that, Following the construction of the prompt words for the intelligent question-answering model based on the target candidate results and the user query, the method further includes: Set constraints; The step of outputting the query results corresponding to the user's query based on the prompt words and using the intelligent question-answering model includes: Based on the constraints and the prompts, the intelligent question-answering model outputs the query results corresponding to the user's query.

6. The intelligent question-answering processing method according to claim 1, characterized in that, The step of converting the data content in the dataset into numerical vectors includes: The data in the dataset is converted into a digital vector using a preset batch size.

7. The intelligent question-answering processing method according to claim 1, characterized in that, After converting the data in the dataset into numerical vectors, the method further includes: The numerical vector is normalized.

8. An intelligent question-and-answer processing device, characterized in that, include: The acquisition module is used to retrieve data information from the dataset. The first construction module is used to construct metadata based on the data information; The conversion module is used to convert the data content in the dataset into digital vectors; The first determining module is used to obtain multiple data records based on the data content, data information, metadata and digital vector corresponding storage in the dataset; The second construction module is used to construct a BM25 search engine based on the data records using the BM25 algorithm. The second determining module is used to obtain, based on the user query, a first candidate result related to the user query and a first relevance score of the first candidate result using the BM25 retrieval tool. The third determining module is used to obtain a second candidate result related to the user query and a second relevance score of the second candidate result based on the query vector corresponding to the user query and the data record. The fourth determining module is used to determine a comprehensive score based on the first correlation score and the second correlation score; The fifth determining module is used to determine the target candidate result based on the comprehensive score; The third construction module is used to construct prompt words for the intelligent question-answering model based on the target candidate results and the user query; The output module is used to output the query results corresponding to the user query based on the prompt words and the intelligent question-answering model.

9. An electronic device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the intelligent question-answering processing method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the intelligent question-answering processing method according to any one of claims 1-7.

Images