Data demand intention analysis and data supply method and system
By identifying and understanding the user's data demand intentions and combining vector calculations to search semantic similarity, the problems of difficult, long cycle and high cost in traditional data supply methods are solved, and efficient, intelligent and convenient data supply is achieved.
Patent Information
- Application Number
- CN202411662933.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-20
- Publication Date
- 2025-03-28
AI Technical Summary
Traditional data supply methods rely on manual collection or fixed-mode reporting systems, which have problems such as difficult, long cycle and high cost in data collection, making it difficult to meet the needs of fast and flexible data acquisition.
By identifying data demand intentions, using a large language model to automatically understand and identify users' data demand intentions, combining vector calculations to search semantic similarity in the vector database, generate query conditions, and perform data query and supply.
It realizes efficient, intelligent and convenient data supply, improves the accuracy of data query, reduces the complexity of user operations, and supports dynamic adjustment of requirements.
Smart Images
Figure CN119719331A_ABST
Abstract
Description
Background Art
[0002] At present, the technologies used for data supply are: manual collection, which requires high professional skills of personnel and has high manpower and time costs. Traditional reporting systems are complex to operate, have high learning costs, have limited flexibility and automation, and have low information acquisition efficiency, making it difficult to meet the needs of fast and flexible data acquisition.
[0003] With the rapid development of artificial intelligence technology, data supply has become a key link for enterprises and institutions to support business decisions, optimize processes and realize intelligent management, such as real-time performance monitoring, product data analysis, human resources statistics, etc. Data supply also plays an indispensable role in helping enterprises gain insight into market trends, optimize product strategies and improve customer service. However, traditional data supply methods rely on manual collection or fixed-mode reporting systems, which have many limitations, and data collection is difficult, time-consuming and costly. Summary of the invention
[0004] The present application provides a data demand intention analysis and data supply method and system to achieve efficient, intelligent and convenient data supply.
[0005] In a first aspect, a data demand intention analysis and data supply method is provided, comprising the following steps:
[0006] Identify the data demand intention to obtain the identified data demand intention;
[0007] Performing semantic vector retrieval on the identified data demand intention to obtain semantic actual indicators and query conditions;
[0008] Based on the semantic actual indicators and query conditions, data is queried and supplied.
[0009] In the above technical scheme, the data demand intention is identified by identifying the data demand intention; the semantic vector is retrieved for the identified data demand intention to obtain the semantic actual indicators and query conditions; based on the semantic actual indicators and query conditions, the data is queried and supplied; the user's data demand intention is automatically understood and identified through a large language model to achieve efficient, intelligent and convenient data supply; the accuracy of data query is improved by vector-based semantic similarity retrieval and matching.
[0010] In a specific embodiment, it also includes:
[0011] Provide knowledge support for data through knowledge base question and answer.
[0012] In a specific implementation scheme, identifying the data demand intention and obtaining the steps of identifying the data demand intention specifically include:
[0013] Build a large model for intent recognition;
[0014] The data demand intent is identified using the intention recognition big model to obtain the identified data demand intent.
[0015] In a specific implementation scheme, the intention recognition large model includes the Qwen2.5 large model.
[0016] In a specific implementation scheme, the steps of performing semantic vector retrieval on the identified data demand intention to obtain semantic actual indicators and query conditions are specifically as follows:
[0017] By vector calculation, a semantic-based similarity search is performed on the identified data requirement intention in the Milvus vector database to obtain the semantic actual index and query conditions.
[0018] In a specific implementation scheme, the steps of querying and supplying data based on the semantic actual indicators and query conditions are specifically as follows:
[0019] Based on the actual semantic indicators and query conditions, SQL is spliced to complete the query and provided to the user to complete the data supply.
[0020] In a specific implementation plan, the steps of providing knowledge support for data through knowledge base question and answer are as follows:
[0021] Use RAG technology to provide knowledge support for data through knowledge base question and answer.
[0022] In a specific implementation scheme, the process of identifying data demand intent supports dynamic adjustment of demand.
[0023] In the second aspect, a data demand intention analysis and data supply system is provided, including:
[0024] A demand intention recognition module is used to recognize the data demand intention and obtain the recognized data demand intention;
[0025] A semantic vector retrieval module, used to perform semantic vector retrieval on the identified data requirement intent to obtain semantic actual indicators and query conditions;
[0026] The query supply module is used to query and supply data based on the actual semantic indicators and query conditions.
[0027] In the above technical scheme, the data demand intention is identified by identifying the data demand intention; the semantic vector is retrieved for the identified data demand intention to obtain the semantic actual indicators and query conditions; based on the semantic actual indicators and query conditions, the data is queried and supplied; the user's data demand intention is automatically understood and identified through a large language model to achieve efficient, intelligent and convenient data supply; the accuracy of data query is improved by vector-based semantic similarity retrieval and matching.
[0028] In a specific embodiment, it also includes:
[0029] The knowledge base question and answer module is used to provide knowledge support for data through knowledge base question and answer. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] Figure 1 A flowchart of a data demand intention analysis and data supply method provided in an embodiment of the present application;
[0031] Figure 2 A structural block diagram of the data demand intention analysis and data supply system provided in an embodiment of the present application. DETAILED DESCRIPTION
[0032] The present application is further described in detail below through the accompanying drawings and embodiments. Through these descriptions, the characteristics and advantages of the present application will become clearer and more specific.
[0033] The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. Although various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless otherwise noted.
[0034] In addition, the technical features involved in different embodiments of the present application described below can be combined with each other as long as they do not conflict with each other.
[0035] To facilitate understanding of the data demand intention analysis and data supply method and system provided by the embodiment of the present application, its application scenario is first explained. The data demand intention analysis and data supply method and system provided by the embodiment of the present application are used to realize efficient, intelligent and convenient data supply. With the rapid development of artificial intelligence technology, data supply has become a key link for enterprises and institutions to support business decision-making, optimize processes and realize intelligent management, such as real-time performance monitoring, product data analysis, human resources statistics, etc. Data supply also plays an indispensable role in helping enterprises to gain insight into market trends, optimize product strategies and improve customer service. However, the traditional data supply method relies on manual collection or fixed-mode reporting systems, which has many limitations, and data collection is difficult, long cycle and high cost. For this reason, the embodiment of the present application provides a data demand intention analysis and data supply method and system to realize efficient, intelligent and convenient data supply. The following is a detailed description of the embodiment in conjunction with the specific drawings.
[0036] refer to Figure 1 and Figure 2 , Figure 1 A flowchart of a data demand intention analysis and data supply method provided in an embodiment of the present application; Figure 2 A structural block diagram of the data demand intention analysis and data supply system provided in an embodiment of the present application.
[0037] exist Figure 1 In the embodiment of the present application, a data demand intention analysis and data supply method is provided, comprising the following steps:
[0038] Identify the data demand intention to obtain the identified data demand intention;
[0039] Performing semantic vector retrieval on the identified data demand intention to obtain semantic actual indicators and query conditions;
[0040] Based on the semantic actual indicators and query conditions, the data is queried and supplied.
[0041] In the above technical scheme, the data demand intention is identified by identifying the data demand intention; the semantic vector is retrieved for the identified data demand intention to obtain the semantic actual indicators and query conditions; based on the semantic actual indicators and query conditions, the data is queried and supplied; the user's data demand intention is automatically understood and identified through a large language model to achieve efficient, intelligent and convenient data supply; the accuracy of data query is improved by vector-based semantic similarity retrieval and matching.
[0042] In a specific embodiment, it also includes:
[0043] Provide knowledge support for data through knowledge base question and answer.
[0044] In the above technical solution, the fusion service of vector computing and big models is combined, and the user's demand intention is extracted from the conversation in the form of natural language dialogue using a big language model. The extraction results are searched based on semantic similarity in the vector database through vector computing, so as to provide data to users quickly, accurately and intelligently. Combining natural language processing and vector computing, the user's operation complexity is reduced through dialogue, the accuracy of data query is improved by semantic vector retrieval, and data requirements can be dynamically adjusted in multiple rounds of dialogue, with diversified chart display forms, which is a flexible, efficient and intelligent data supply method.
[0045] In a specific implementation scheme, identifying the data demand intention and obtaining the steps of identifying the data demand intention specifically include:
[0046] Build a large model for intent recognition;
[0047] The data demand intent is identified using the intention recognition big model to obtain the identified data demand intent.
[0048] In a specific implementation scheme, the intention recognition large model includes the Qwen2.5 large model.
[0049] In a specific implementation scheme, the steps of performing semantic vector retrieval on the identified data demand intention to obtain semantic actual indicators and query conditions are specifically as follows:
[0050] By vector calculation, a semantic-based similarity search is performed on the identified data requirement intention in the Milvus vector database to obtain the semantic actual index and query conditions.
[0051] In a specific implementation scheme, the steps of querying and supplying data based on the semantic actual indicators and query conditions are specifically as follows:
[0052] Based on the actual semantic indicators and query conditions, SQL is spliced to complete the query and provided to the user to complete the data supply.
[0053] In a specific implementation plan, the steps of providing knowledge support for data through knowledge base question and answer are as follows:
[0054] Use RAG technology to provide knowledge support for data through knowledge base question and answer.
[0055] In a specific implementation scheme, the process of identifying data demand intent supports dynamic adjustment of demand.
[0056] Specifically, the data demand intention analysis and data supply method is a Python application developed using LangChain and LangGraph that combines vector computing and large models. It receives user conversations through a page, maintains multiple rounds of conversations corresponding to each conversation in the background, uses the Qwen2.5 large model to extract user data demand intentions from multiple rounds of conversations, and semantically retrieves the closest actual indicators and query conditions in the Milvus data vector library; splices SQL to execute queries and generates tables, line graphs, and pie charts required by users; at the same time, through RAG technology, uses vector libraries and large models to provide users with knowledge base question-and-answer services related to data collection. Specifically, it includes the following steps:
[0057] Step 1: Intent recognition:
[0058] The data supply agent is built based on the Qwen2.5 big model. It has two functions / tools that can be called: data collection and help. Based on the conversation type, the agent's call to the tool will logically form three routes: data collection, help (knowledge base Q&A), and chat. When the user comes to collect data: first use the big model to cut and summarize historical conversations to prevent the long conversation from affecting the accuracy of the big model response, and then extract the user's indicator requirements (such as performance trends by institution, performance by insurance type, etc.), query conditions (such as institutions, institutional levels, channels, statistical dates, etc.), and display forms (trends, proportions, tables, line charts, pie charts);
[0059] Step 2: Semantic vector retrieval:
[0060] Through vector calculation, the big model extraction results are searched based on semantic similarity in the Milvus vector database to obtain the actual indicators and query conditions with the closest semantics. At the same time, the latest status of these key information in the current conversation is maintained. When the big model does not extract the query conditions or the vector search does not have similar results, it falls back to this state to prevent the big model from not extracting the information mentioned by the user in the previous rounds of conversations, resulting in inaccurate data supply, and improve the coherence of multiple rounds of conversations;
[0061] Step 3: Data query and display:
[0062] By combining vector computing and large model information extraction, we obtain the indicators, query conditions, and display formats that users want. We then splice SQL to complete the query and provide it to users in the form of tables, line graphs, pie charts, etc., to complete the data supply.
[0063] Step 4: Knowledge Base Q&A:
[0064] The knowledge base related to data collection often contains rich information, including data definitions, indicator explanations, data usage guidelines, etc. After the knowledge is sorted out, it is cut into text fragments and uploaded to the Milvus vector database. When users ask questions related to data collection, RAG technology is used to condense multiple rounds of dialogue into one sentence through the big model. The knowledge fragments related to the condensed results are retrieved in the Milvus vector database through vector calculation. Subsequently, these search results are provided to the big model as prompt words together with the user's questions to help the big model generate answers, provide users with more intuitive and immediate knowledge support, and enable users to better understand the meaning of data and indicator background.
[0065] In the above technical solutions, 1. Natural language dialogue is adopted: through a large language model, the user's data demand intention is automatically understood and identified, realizing efficient, intelligent and convenient data supply; 2. Vector-based semantic similarity retrieval and matching: improving the accuracy of data query; 3. Dynamic adjustment of demand: users can dynamically adjust data requirements in multiple rounds of dialogues, and each updated demand will instantly return relevant data, significantly improving the flexibility of data supply; 4. Knowledge base question and answer function: providing users with knowledge base question and answer services related to data collection. Users can consult data meaning, indicator explanation and other information through dialogue, making the data query process more intelligent.
[0066] exist Figure 2 In the embodiment of the present application, a data demand intention analysis and data supply system is provided, including:
[0067] A demand intention recognition module is used to recognize the data demand intention and obtain the recognized data demand intention;
[0068] A semantic vector retrieval module, used to perform semantic vector retrieval on the identified data requirement intent to obtain semantic actual indicators and query conditions;
[0069] The query supply module is used to query and supply data based on the actual semantic indicators and query conditions.
[0070] In the above technical scheme, the data demand intention is identified by identifying the data demand intention; the semantic vector is retrieved for the identified data demand intention to obtain the semantic actual indicators and query conditions; based on the semantic actual indicators and query conditions, the data is queried and supplied; the user's data demand intention is automatically understood and identified through a large language model to achieve efficient, intelligent and convenient data supply; the accuracy of data query is improved by vector-based semantic similarity retrieval and matching.
[0071] In a specific embodiment, it also includes:
[0072] The knowledge base question and answer module is used to provide knowledge support for data through knowledge base question and answer.
[0073] Those skilled in the art will appreciate that the present application may be implemented as a system, method or computer program product.
[0074] Therefore, the present disclosure may be specifically implemented in the following forms, namely: it may be completely hardware, it may be completely software (including firmware, resident software, microcode, etc.), or it may be a combination of hardware and software, generally referred to herein as a "circuit", "module" or "system". In addition, in some embodiments, the present application may also be implemented in the form of a computer program product in one or more computer-readable media, and the computer-readable medium may contain computer-readable program code.
[0075] Any combination of one or more computer-readable media can be used. Computer-readable media can be computer-readable signal media or computer-readable storage media. Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: electrical connections with one or more wires, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this document, computer-readable storage media can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.
[0076] Although the embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art can change, modify, replace and modify the above embodiments within the scope of the present application. On this basis, a variety of replacements and improvements can be made to the present application, all of which fall within the scope of protection of the present application.
Claims
1. A data demand intention analysis and data supply method, characterized in that: The following steps are involved: Identify the data demand intention to obtain the identified data demand intention; Performing semantic vector retrieval on the identified data demand intention to obtain semantic actual indicators and query conditions; Based on the semantic actual indicators and query conditions, data is queried and supplied.
2. The data demand intention analysis and data supply method according to claim 1, characterized in that: Also includes: Provide knowledge support for data through knowledge base question and answer.
3. The data demand intention analysis and data supply method according to claim 2, characterized in that: The data demand intention is identified, and the steps for identifying the data demand intention are obtained, which specifically include: Build a large model for intent recognition; The data demand intent is identified using the intention recognition big model to obtain the identified data demand intent.
4. The data demand intention analysis and data supply method according to claim 3 is characterized in that: The intention recognition large model includes the Qwen2.5 large model.
5. The data demand intention analysis and data supply method according to claim 4 is characterized in that: The steps of performing semantic vector retrieval on the identified data demand intention to obtain semantic actual indicators and query conditions are specifically as follows: By vector calculation, a semantic-based similarity search is performed on the identified data requirement intention in the Milvus vector database to obtain the semantic actual index and query conditions.
6. The data demand intention analysis and data supply method according to claim 5, characterized in that: The steps of querying and supplying data based on the semantic actual indicators and query conditions are specifically as follows: Based on the actual semantic indicators and query conditions, SQL is spliced to complete the query and provided to the user to complete the data supply.
7. The data demand intention analysis and data supply method according to claim 6, characterized in that: The steps to provide knowledge support for data through knowledge base question and answer are as follows: Use RAG technology to provide knowledge support for data through knowledge base question and answer.
8. The data demand intention analysis and data supply method according to claim 7, characterized in that: The process of identifying data demand intent and supporting dynamic adjustment of demand.
9. A data demand intention analysis and data supply system, characterized in that: include: A demand intention recognition module is used to recognize the data demand intention and obtain the recognized data demand intention; A semantic vector retrieval module, used to perform semantic vector retrieval on the identified data requirement intent to obtain semantic actual indicators and query conditions; The query supply module is used to query and supply data based on the actual semantic indicators and query conditions.
10. The data demand intention analysis and data supply system according to claim 9, characterized in that: Also includes: The knowledge base question and answer module is used to provide knowledge support for data through knowledge base question and answer.