Intelligent BI access method and device and readable storage medium

Abstract

The invention provides an intelligent BI access method and device and a readable storage medium, and the method comprises the steps: obtaining a user question, and determining a target intention most related to the user question through a large language model; based on the target intention, determining a corresponding task description Prompt; performing parameter optimization processing on the task description Prompt through a small model to obtain an optimized Prompt; and performing searching and access based on the optimized Prompt to obtain corresponding target data. According to the intelligent BI access method and device and the readable storage medium provided by the invention, the index parameters, the time parameters, the address parameters and the like extracted by the large language model are converted and corrected through the small model to obtain the standard parameters required for access of the application system interface, so that rapid and accurate intelligent BI access is realized.

Description

Technical Field

[0001] The present application relates to the field of Internet technology, and in particular to an intelligent BI data acquisition method, device and readable storage medium. Background Art

[0002] BI data extraction, or Business Intelligence data extraction, refers to the process of extracting data from various data sources for analysis and processing. It is a key step in the business intelligence system. The purpose of BI data extraction is to transform raw data into valuable information to help companies make more informed decisions.

[0003] At present, the BI software market is growing continuously and is widely used in various enterprises, especially in data analysis and decision support. The functions of BI software are constantly enhanced, including data warehouse platform, BI platform software and BI application software. These technologies help enterprises achieve data integration and unified management, and then transform data into useful information and knowledge. In addition, the application scope of BI is constantly expanding, not only limited to traditional data report generation, but also including data exploration, analysis and prediction.

[0004] However, due to the complexity of the BI data acquisition process, BI data acquisition still has certain shortcomings in terms of result accuracy and timeliness. Summary of the invention

[0005] The technical problem to be solved by the present application is to provide an intelligent BI data acquisition method, device and readable storage medium to solve the problems existing in the prior art in view of the above-mentioned deficiencies in the prior art.

[0006] In a first aspect, the present application provides an intelligent BI data acquisition method, the method comprising:

[0007] S1. Obtain user questions and determine the target intent most relevant to the user questions through a large language model;

[0008] S2. Based on the target intention, determine the corresponding task description Prompt;

[0009] S3, performing parameter optimization processing on the task description Prompt through the small model to obtain an optimized Prompt;

[0010] S4. Search and retrieve data based on the optimized Prompt to obtain corresponding target data.

[0011] In some embodiments, determining the target intent most relevant to the user question through a large language model includes:

[0012] Performing intent recognition on the user question through a large language model to obtain an intent recognition result;

[0013] Filtering out a target intent most relevant to the user question from a plurality of preset intents according to the intent recognition result;

[0014] Among them, the preset intention is pre-set based on the demand scenario.

[0015] In some embodiments, S2 includes:

[0016] Perform parameter parsing on the user question through a large language model to obtain a parameter parsing result;

[0017] According to the parameter analysis result, the target task most relevant to the user question is selected from a plurality of preset tasks corresponding to the target intent, and a task description prompt corresponding to the target task is determined;

[0018] Wherein, the preset task is pre-set based on the preset intention.

[0019] In some embodiments, S3 includes:

[0020] Based on the task description Prompt, extract parameters of the user question through a large language model to obtain indicator parameters;

[0021] Using a small model, select the target parameter that is most similar to the indicator parameter from the standard parameter name library;

[0022] The optimized prompt is obtained based on the target parameter that is most similar to the indicator parameter.

[0023] In some embodiments, the target parameter most similar to the indicator parameter is selected from the standard parameter name library by the small model, including:

[0024] Using the NLTK edit distance similarity matching model, multiple standard parameters similar to the indicator parameters are selected from the standard parameter name library;

[0025] Through a deep neural network similarity matching model, a target parameter most similar to the indicator parameter is selected from a plurality of standard parameters similar to the indicator parameter.

[0026] In some embodiments, S3 further includes:

[0027] According to the user's question, the prompt description of the time parameter extraction task is optimized, and the optimized time parameters are obtained based on the optimized prompt.

[0028] In some embodiments, it also includes:

[0029] Semantic splicing is performed based on the target data obtained from the data collection to obtain the answer text corresponding to the user's question.

[0030] In a second aspect, the present application provides an intelligent BI data acquisition device, the device comprising:

[0031] An intent determination module configured to obtain a user question and determine a target intent most relevant to the user question through a large language model;

[0032] A prompt determination module, which is configured to determine a corresponding task description prompt based on the target intention;

[0033] A parameter optimization module, which is configured to perform parameter optimization processing on the task description Prompt through a small model to obtain an optimized Prompt;

[0034] The search and data acquisition module is configured to perform search and data acquisition based on the optimized Prompt to obtain corresponding target data.

[0035] In a third aspect, the present application provides an intelligent BI data acquisition device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to implement the intelligent BI data acquisition method described in the first aspect above.

[0036] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the intelligent BI data acquisition method described in the first aspect is implemented.

[0037] The present application provides an intelligent BI data acquisition method, device and readable storage medium, the method includes: obtaining user questions, and determining the target intent most relevant to the user questions through a large language model; determining the corresponding task description prompt based on the target intent; performing parameter optimization processing on the task description prompt through a small model to obtain an optimized prompt; searching and acquiring data based on the optimized prompt to obtain the corresponding target data. The present application proposes an intelligent BI data acquisition method, device and readable storage medium, which converts and corrects the indicator parameters, time parameters and address parameters extracted by the large language model through a small model to obtain the standard parameters required for data acquisition by the application system interface, thereby achieving fast and accurate intelligent BI data acquisition. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0039] Figure 1 A flowchart of an intelligent BI data acquisition method provided in an embodiment of the present application;

[0040] Figure 2 A flowchart of another intelligent BI data acquisition method provided in an embodiment of the present application;

[0041] Figure 3 A schematic diagram of the structure of an intelligent BI data acquisition device provided in an embodiment of the present application;

[0042] Figure 4 A schematic diagram of the structure of another intelligent BI data acquisition device provided in an embodiment of the present application.

[0043] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0044] In order to enable those skilled in the art to better understand the technical solution of the present application, the implementation mode of the present application will be further described in detail below with reference to the accompanying drawings.

[0045] It should be understood that the specific embodiments and drawings described herein are only used to explain the present application, rather than to limit the present application.

[0046] It can be understood that, in the absence of conflict, the various embodiments in the present application and the various features in the embodiments can be combined with each other.

[0047] It can be understood that, for the convenience of description, the drawings of the present application only show the parts related to the present application, while the parts unrelated to the present application are not shown in the drawings.

[0048] It can be understood that each unit and module involved in the embodiments of the present application may correspond to only one physical structure, or may be composed of multiple physical structures, or multiple units and modules may be integrated into one physical structure.

[0049] It can be understood that the terms "first", "second", etc. in the embodiments of the present application are used to distinguish different objects, or to distinguish different processing of the same object, rather than to describe a specific order of objects.

[0050] It is understandable that, in the absence of conflict, the functions and steps marked in the flowcharts and block diagrams of the present application may occur in an order different from that marked in the drawings.

[0051] It is understood that the flowcharts and block diagrams of the present application illustrate the possible architectures, functions, and operations of the systems, devices, equipment, and methods according to the various embodiments of the present application. Among them, each box in the flowchart or block diagram may represent a unit, module, program segment, code, which contains executable instructions for implementing the specified functions. Moreover, each box or combination of boxes in the block diagram and flowchart may be implemented by a hardware-based system that implements the specified functions, or by a combination of hardware and computer instructions.

[0052] It can be understood that the units and modules involved in the embodiments of the present application can be implemented by software or hardware, for example, the units and modules can be located in a processor.

[0053] At present, some existing technologies use large language models to perform data acquisition processing.

[0054] The Big Language Model is a powerful natural language processing (NLP) tool that has emerged in the field of artificial intelligence in recent years. It is based on the Transformer architecture and is a large-scale neural network model trained with a large amount of text data. The Big Language Model has strong natural language understanding and generation capabilities, including semantic understanding, intent recognition, logical reasoning, sentiment analysis, parameter extraction and other very rich and practical natural language processing capabilities.

[0055] With the popularity of ChatGPT, a wave of big language models has been set off in the field of artificial intelligence. More and more artificial intelligence practitioners and researchers have focused their work and research on big language models. Basic big language models based on general fields and industry big language models based on various vertical fields are emerging in an endless stream. The continuous release of large language models is dazzling, and the open source big language model community is also thriving.

[0056] As big language model technology continues to make new breakthroughs, more and more companies are beginning to pay attention to how to apply the results of big language model technology to production practice in order to improve production efficiency, reduce production and operation costs, and improve service levels and product quality.

[0057] At present, the scale of telecommunications operators' networks is huge and the structure is complex. The pressure to ensure stable network operation and good network service quality is very high. Every link of network planning, construction, maintenance, optimization and operation is very complex and time-consuming, requiring a lot of manpower costs, which brings huge challenges and burdens to operators' daily production and operation. How to apply large language models to the production and operation of operators' networks and help the network evolve into a self-intelligent network is a key issue for operators to consider.

[0058] Currently, operators have tried to apply big language model technology to the links of network intelligent operation, such as using big language model to realize intelligent question and answer of network problems, intelligent data collection of network indicators, SQL generation, network fault analysis (AIAgent), etc. However, as big language model technology is still in the process of rapid development, although new breakthroughs are constantly being made, big language model technology is not mature enough and has certain limitations, such as occasional hallucinations, weak mathematical and logical reasoning ability, and poor ability to understand long texts.

[0059] As mentioned in the previous background technology, although large language models have made continuous breakthroughs and have been widely applied in the telecommunications industry, the current large language model technology still has certain limitations. In the operator's network intelligent operation scenario, operators need to follow strict security and compliance standards. Even a small error may cause the failure of important indicators to be obtained, which in turn affects the key parameter configuration of network equipment and causes network failures. In serious cases, it may have a large-scale negative social impact.

[0060] Specifically, the defects of large language models in the prior art mainly include the following points:

[0061] 1. In the indicator query task, the indicator name parameters extracted by the large language model are difficult to meet expectations, and the semantic understanding and generation capabilities need to be improved.

[0062] For example, a user's question is: "Query the target of Beijing's full port utilization in March 2024". The indicator name parameter that the large language model should extract is "full port utilization", but the actual indicator name parameter extracted by the large language model is "Beijing's full port utilization", and sometimes even "Beijing's full port utilization target". If you use the indicator name extracted by the large language model to apply the system interface to obtain data, it is obvious that the corresponding data cannot be obtained because the indicator parameter name cannot be matched in the system.

[0063] It can be seen that due to the defects of the large language model itself, irrelevant information such as addresses are mixed into the indicator name when extracting business indicator parameters, thus affecting the accuracy of the data extraction results.

[0064] 2. The large language model often fails to achieve the expected results in time parameter extraction tasks involving logical reasoning, and its mathematical and logical reasoning capabilities need to be improved.

[0065] For example, the user's question is: "Query the number of 5G base stations in Shanghai in the past 10 months." The large language model needs to extract time, location, and indicator name parameters. For time parameter extraction, although the task prompt tells the large language model: "The start time is before the end time, and the current time is July 2024. If you query the past 10 months, the end time is the current time, and the start time is 10 months before the end time." The time parameters that the large language model should extract are: start time: September 2023, end time: July 2024. However, the parameters actually obtained by the large language model are often incorrect. Sometimes the difference between the start time and the end time is not 10 months, and sometimes the start time and the end time are reversed, resulting in the end time being earlier and the start time being later.

[0066] It can be seen that due to the defects of the large language model itself, the extraction time parameters do not conform to the natural language logic, thus affecting the accuracy of the data extraction results.

[0067] 3. Since user input questions are subjective and habitual, for rigorous system interfaces that do not provide fuzzy query capabilities, the business indicator parameters extracted by the large language model cannot be directly retrieved through the application system.

[0068] For example, the user's question is: "Query the number of 5G base stations in Beijing in March 2024". The large language model extracts the correct indicator name parameter: "5G base station number", but the actual application system interface standard parameters may not have this indicator name, only "5G medium frequency base station number". At this time, if you use the indicator name extracted by the large language model to retrieve data in the application system, you will not be able to retrieve the corresponding data. In fact, from a subjective point of view, most users do not know that they should ask "5G medium frequency base station number" to query the desired indicator data. Even those who are engaged in related professional work may enter "5G base station number" to query the desired data for convenience or habit. For similar questions, users may also ask indicator query sentences with auxiliary words such as "5G base station number" and "port utilization". In these cases, although the indicator name parameters extracted by the large language model conform to the natural language logic, they cannot match the business indicator parameter name of the business system, resulting in failure to retrieve data.

[0069] It can be seen from this that in practice, due to user habits, the business indicator parameter name may not meet the requirements, resulting in the inability to obtain data or incorrect data collection, which affects the accuracy of the data collection results.

[0070] 4. Some application system interfaces have agreed on a set of standard address names. Large language models cannot strictly output according to the standard address names when extracting parameters, and thus cannot obtain data from the application system interface.

[0071] This situation mainly occurs in addresses. For example, some systems stipulate that the address standard is the full name, such as Shanghai, Guangxi Zhuang Autonomous Region, etc., but when users ask questions, most of them may directly ask: "Query a certain indicator of Central Mongolia at a certain time" or "What is a certain indicator in Guangxi at a certain time?" In this case, although the large language model extracts the name of the address province, it does not meet the interface standard specification requirements and cannot be directly retrieved through the application system interface.

[0072] It can be seen from this that in practice, due to user habits, the address indicator parameter name may not meet the requirements, resulting in the inability to obtain data or incorrect data acquisition, which affects the accuracy of the data acquisition results.

[0073] Therefore, in the actual production implementation process, the large language model technology also needs to rely on some small model capabilities, large language model orchestration and scheduling strategies, pre- and post-processing methods, and other methods to assist in processing user input problems, so that the large language model can understand user intentions and output expected parameters.

[0074] In order to solve the problem of incorrect extraction of indicator name parameters, incorrect extraction of time parameters, and inconsistency between indicator parameters and address parameters and standard parameter names during the parameter extraction process of the large language model, which leads to the inability to extract user expected indicator data, the present application proposes an intelligent BI data acquisition method, device and readable storage medium, which converts and corrects the indicator parameters, time parameters and address parameters extracted by the large language model through a small model to obtain the standard parameters required for data acquisition by the application system interface, thereby realizing fast and accurate intelligent BI data acquisition.

[0075] The main ideas of this application are:

[0076] First, use the large language model to filter out the intent that is closest to the user's question from several intents; then, let the large language model extract the corresponding parameters from the user's question according to the task description prompt corresponding to the intent, such as network business indicators, application name, professional name, address, time, etc.; then, use the NLP similarity matching small model to search according to the standard parameter table, match the most similar standard parameters, and then use the time processing small model to correct the errors in time parameter extraction and obtain the standard parameters of the application system interface; finally, supplement the work parameters according to the intent, and convert various parameters into json through post-processing, and then bring the obtained json into the application system interface to obtain data and display it on the front-end page.

[0077] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0078] The present application provides an intelligent BI data acquisition method, the working process of which can be implemented by electronic devices, such as computers, handheld smart terminals, etc. For the convenience of explanation, the embodiments of the present application are described with the method execution subject being a computer.

[0079] Figure 1 A schematic diagram of the intelligent BI data acquisition method provided in the embodiment of the present application, such as Figure 1 As shown, in view of the deficiencies of the prior art, this application proposes an intelligent BI data acquisition method, device and readable storage medium, the main steps of which are:

[0080] First, define full intent and full tasks based on the demand scenario;

[0081] Then, we identify the intent based on the user's question and select the specific intent that matches the user's question from all the intents.

[0082] Next, get the specific task according to the intent number, execute the task according to the task description prompt and optimize the prompt to get the basic expected indicator name, time, address and other parameters;

[0083] Then enter post-processing, add other necessary parameters, modify content, convert format, and obtain standard JSON parameters that can be parsed by the backend;

[0084] Then, the backend retrieves data through the specific application system interface, performs semantic splicing to obtain the answer that the expected user wants, and forwards it to the front-end development.

[0085] Finally, the front end displays the results to the APP question and answer window.

[0086] The step of defining full intent and full tasks according to the demand scenario can be performed before obtaining user questions. The specific content of this step is as follows:

[0087] 1. Determine the demand scenario

[0088] In the question-and-answer module of a network APP that contains several network sub-scenario applications, network indicator data query is implemented through a large language model. When an APP user asks questions related to indicator data query in the question-and-answer window, the large language model identifies the intention based on the user's question analysis and extracts relevant parameters, then calls the corresponding system interface to obtain the corresponding data, and returns the expected answer to the APP user through the question-and-answer window, thereby realizing intelligent BI data acquisition. The scenarios involved in intelligent BI data acquisition include indicator data query for several scenarios such as network indicators, network resources, fault alarm information, and system work order information.

[0089] 2. Define the full intent

[0090] According to the demand scenarios and the goals that the large language model needs to accomplish, define the full amount of intents that the large language model needs to recognize, and number all intents sequentially. The full amount of intents for network-related indicator queries includes independent indicator queries for various scenarios, changes in dimensions such as time, location, and trends, and combined indicator queries. For example, query monthly granularity network indicator data, daily granularity network indicator data, indicator changes compared to the previous month, month-on-month indicator changes, pending / completed work orders, professional pending / completed work orders, application pending / completed work orders, etc. There are more than 600 types of network indicators here, more than 150 types of sub-scenario applications corresponding to work orders, 10 types of professional types corresponding to work orders (such as wireless networks, core networks, broadband networks, etc.), and there are also many network resource indicators.

[0091] For example, the following are some sample intents:

[0092] 1. Monthly granularity indicator data, query the indicator value of a certain month, such as this month, June 2024

[0093] 2. Daily granularity indicator data, asking for the value of a certain indicator on a certain day, such as today or June 1, 2024

[0094] 3. Changes in monthly granularity index numbers compared to month-on-month values, such as this month, June, and June 2024

[0095] 4. Monthly granularity indicator data targets, ask for indicator targets, time such as this month, June, June 2024

[0096] 5. Monthly granularity indicator data target completion rate, ask the indicator target completion rate, time such as this month, June, June 2024

[0097] 6. The number changes / trends of indicators with monthly granularity in recent months, such as January to May, the past 10 months, etc.

[0098] 7. The number of indicators in recent months with monthly granularity, and the indicator values ​​within the time period, such as January to May, the past year, etc.

[0099] 8. Regional distribution of indicators with monthly granularity, asking for regional distribution, time, for example: this month, June 2024

[0100] 9. Daily granularity index proportion by province, ask about the proportion by province, time, for example: today, June 1, 2024

[0101] 10. Daily granularity to-do work orders, with time such as now, today, and June 1, 2024

[0102] 11. To-do work orders under a certain application name

[0103] 12. To-do work orders under a work order specialty

[0104] 13. Work order number / ID is XXXX

[0105] 14. Information about a computer room at a certain address

[0106] 15. Information about the computer room with the ID number / ID of XXXX

[0107] 3. Define full tasks based on full intent

[0108] After determining all intents and numbering them, you need to define the corresponding tasks according to the intents. Here, you mainly extract parameters from user questions according to the input parameter requirements of each actual business system interface. Parse parameters based on user questions, such as time, location, indicator name, etc., and add additional parameters based on the intent. For example, you can determine the additional relevant parameters based on whether relevant answer text, trend charts, link jumps, etc. are provided.

[0109] After defining all the tasks, number the tasks, and correspond the intent and task description numbers one by one, so that after determining the specific intent, you can select the corresponding specific task according to the intent number.

[0110] For example, the following is a specific task prompt corresponding to a certain intent. When the large language model recognizes that the intent is "query the changes in the monthly granularity indicator data in the past month", the task corresponding to the intent is executed and the corresponding json is returned. Here, since the prompt contains the system parameter now_month, which is the month corresponding to the current time of the server system, in order to avoid errors caused by json format conflicts, {{ and}} are used here to replace { and}. In the figure, filed, method, and showWay are fixed supplementary parameters, which need to be supplemented according to the intent and the back-end interface data acquisition. The parameters such as start time, end time, province, and indicator name are obtained from user questions.

[0111] The content between f""" and """ below is a Prompt example of a task processing using a large language model:

[0112] f"""Query the changes of monthly granularity indicator data in the past month

[0113] Parameter extraction: Parameters include monthly granularity time, provincial address, and indicator name. Assume that the query is for the past 18 months, the end time is {nowmonth}, and the start time is 10 months before the end time.

[0114] Parameter format: check time, location and indicator name. The time format is yyyymm, yyyy represents year, mm represents month, the sample is 202401, and the current time is {now_month}; location to provincial level, including province, municipality, autonomous region, special administrative region, such as Anhui and Beijing, indicator name is xx number, xx amount, xx completion rate, etc., such as the number of 5G medium-frequency base stations

[0115] Please use the following json format to give the extracted parameters. Please strictly output in json format. Please fill in None for parameters not mentioned in the question.

[0116] {{

[0117] "type":"BI"

[0118] 'params":[

[0119] {{

[0120] "Start time": xx,

[0121] "End time": xx,

[0122] "Province": xx,

[0123] "Indicator name": xx,

[0124] "field":"value",

[0125] "method":"query"

[0126] }} ]

[0128] "showWay":"fold"

[0129] }}"""

[0130] This application defines the full amount of intents and tasks that the large language model needs to recognize, and numbers them in sequence, based on the demand scenarios and the goals that the large language model needs to accomplish. The large language model selects the most matching intent from the full amount of intents based on the user's question, and then obtains the specific task based on the one-to-one correspondence between the intent number and the task number. The large language model then extracts parameters based on the specific task description. In this way, the amount of text involved in each task execution by the large model will be greatly reduced, thereby greatly reducing the difficulty of the large model's semantic understanding of long texts, while avoiding the problem of multiple places in the same text with similar semantics but different actual intents.

[0131] Figure 2 Another schematic diagram of the intelligent BI data acquisition method provided in the embodiment of the present application is as follows Figure 2As shown, the present application provides an intelligent BI data acquisition method, the method includes S1-S4, which are as follows:

[0132] S1. Obtain user questions and determine the target intent most relevant to the user questions through a large language model;

[0133] In some embodiments, in S1, the target intent most relevant to the user question is determined through a large language model, including: performing intent recognition on the user question through a large language model to obtain an intent recognition result; screening out the target intent most relevant to the user question from a number of preset intents based on the intent recognition result; wherein the preset intent is pre-set based on the demand scenario, corresponding to the full intent in the previous text.

[0134] S2. Based on the target intention, determine the corresponding task description Prompt;

[0135] In some embodiments, S2 includes: performing parameter parsing on the user question through a large language model to obtain a parameter parsing result; based on the parameter parsing result, screening out the target task most relevant to the user question from a number of preset tasks corresponding to the target intent, and determining the task description Prompt corresponding to the target task; wherein the preset task is pre-set based on the preset intent, corresponding to the full task in the previous text.

[0136] S3, performing parameter optimization processing on the task description Prompt through the small model to obtain an optimized Prompt;

[0137] In some embodiments, S3 includes: based on the task description Prompt, extracting parameters of the user question through a large language model to obtain indicator parameters; screening target parameters most similar to the indicator parameters from a standard parameter name library through a small model; and obtaining an optimized Prompt based on the target parameters most similar to the indicator parameters.

[0138] Specifically, after intent recognition and task matching, the large language model clearly knows the specific task it needs to do. The large language model extracts parameters based on user questions and task description prompts.

[0139] For example, the following is an example of a large language model response. The user asked: How has the number of 5G base stations in Shanghai changed in the past 10 months? Combined with the server system time (the current time is July 2024), the large language model infers that the start time is September 2023, the end time is July 2024, the location is Shanghai, and the indicator name is the number of 5G base stations. And output json according to the format agreed in the task prompt, out is the reply result of the large language model, and the following is a preliminary result sample of the user question and the output parameters of the large language model:

[0140] query: How has the number of 56 base stations in Shanghai changed in the past 10 months?

[0141] out:

[0142] ```json

[0143] {

[0144] "type":"BI",

[0145] "params":[

[0146] {

[0147] "Start time":"202309"

[0148] "End time":"202407"

[0149] "Province":"Shanghai"

[0150] "Indicator name": "Number of 5G base stations"

[0151] "field":"value"

[0152] "method":"query"

[0153] } ]

[0155] "showway":"fold"

[0156] }```

[0157] It can be seen that there are some irrelevant characters in the result sample, such as json and the symbol ```, and the indicator name does not conform to the application system interface standard for data acquisition. Although the start time and end time parameters here are correct, they often do not meet expectations in experiments. For example, the output start time is 202310, 202312, etc.

[0158] Optionally, the method further includes executing a step of optimizing Prompt, specifically, determining whether the parameters output by the large language model meet expectations, and if not, adjusting the task Prompt so that the large language model basically meets the output required by the business. If the output meets expectations, the next step of indicator parameter conversion is entered.

[0159] In some embodiments, indicator parameter conversion is implemented through an open source small model. Specifically, the small model is used to select the target parameter most similar to the indicator parameter from the standard parameter name library, including: using the NLTK edit distance similarity matching model to select multiple standard parameters similar to the indicator parameter from the standard parameter name library; using the deep neural network similarity matching model to select the target parameter most similar to the indicator parameter from multiple standard parameters similar to the indicator parameter.

[0160] After the large language model identifies the most relevant question to the user among many intents, it can perform the corresponding task processing: parameter extraction. The following four problems often occur during the parameter extraction process of network business indicators, application names, professional names, addresses and other parameters: 1) The large language model extracts indicator parameters and time parameters incorrectly, 2) The indicator parameters are extracted correctly but are inconsistent with the standard names, 3) The indicator name in the query contains auxiliary words, such as the user question: Query the utilization rate of all ports in Beijing in March 2024, 4) The address name in the query may be an abbreviation or colloquialism, such as "Inner Mongolia" and "Beijing", while the standard interface input needs to be "Inner Mongolia Autonomous Region" and "Beijing City". These problems will result in the inability to extract the indicator data expected by the user because the parameters do not meet the application system interface standards, and thus the BI cannot be realized and can only retrieve data.

[0161] Since the number of indicators contained in the standard parameter list is very large, it takes a lot of time to match with an accurate deep neural network. Combined with the intention selection and task execution process made by the previous large language model, it will cause the overall task response to time out. Therefore, two open source small models are used to convert the indicator name, address, professional name and other parameters output by the large language model into the standard parameters required by the corresponding system interface, thereby realizing intelligent BI data acquisition. First, use "Small Model 1" (NLTK edit distance similarity matching) to quickly screen several approximate similar indicator lists, and then use "Small Model 2" (deep neural network similarity matching) to perform accurate similar indicator matching to obtain the relevant parameters that best match the application system interface standard.

[0162] This application extracts parameters through a large model, and then uses a small model to assist in processing and correcting parameters, avoiding the problem of multiple matches of multiple words and possible confusion of multiple keywords when using only a small NLP word segmentation model, and also avoiding the problems of inaccurate parameter extraction and indicator names not meeting the business data system interface standard when only using a large model to extract parameters. The large model extracts parameters for the first time, and then uses two small models to recall parameter similarity in turn, which can greatly shorten the time for intent recognition and task execution, and ensure the accuracy and availability of the output results.

[0163] In this application, the first indicator parameter conversion process is performed first.

[0164] Specifically, through the edit distance metric provided by NLTK, a small number of parameters similar to the problem are screened out from the candidate parameter name library. The advantage of this method is that it is simple and fast.

[0165] For example, the following is the initial conversion of parameters, NLTK edit distance similarity matching, the user question is: "Query the trend of the number of 5G base stations in Beijing in April 2024", out0 is the parameter indicator name extracted by the large language model: "Number of 5G base stations in Beijing", which can be seen to be somewhat different from the actual indicator. The standard parameter list is about 600+ network indicator parameters in words_lst. After the small model NLTK edit distance similarity matching, the initial conversion results in 4 similar indicators, out1: 2G base station number, 3G base station number, 4G base station number, 5G medium frequency base station number.

[0166] Then, the second indicator parameter conversion process is performed. Specifically, the closest indicator out2: the number of 5G medium frequency base stations is obtained by matching through a text similarity algorithm based on a deep neural network.

[0167] In this application, during the parameter conversion process, two small models are used to perform text similarity analysis to obtain the converted standard parameters that meet the requirements of the application system interface. The main reason is that the number of indicators contained in the standard parameter list is very large. If a more accurate deep neural network is used for matching, it will take a lot of time. Combined with the intention selection and task execution process made by the previous large language model, the overall task response time will be too long. Through small model 1 (NLTK edit distance similarity matching), several approximate similarity indicator lists are quickly screened, and then accurate similarity indicator matching is performed through small model 2 (deep neural network similarity matching) to obtain the parameter indicator name that best meets the user's intention and meets the standard parameters of the application system interface. It can greatly shorten the time for intent recognition and task execution, and ensure the accuracy and availability of the output results.

[0168] In some embodiments, S3 further includes: performing Prompt description optimization processing on the time parameter extraction task according to the user question, and obtaining optimized time parameters based on the optimized Prompt.

[0169] DescriptionFor example, the user's question is: "Query the number of 5G base stations in Shanghai in the past 10 months." The large language model needs to extract the time, location, and indicator name parameters. For the extraction of time parameters, although the task prompt tells the large language model: "The start time is before the end time, and the current time is July 2024. If the query is in the past 10 months, the end time is the current time, and the start time is 10 months before the end time." The time parameters that the large language model should extract are: start time: September 2023, end time: July 2024. However, the parameters actually obtained by the large language model are often incorrect. Sometimes the difference between the start time and the end time is not 10 months, and sometimes the start time and the end time are reversed, resulting in the end time being in front and the start time being in the back.

[0170] Therefore, this step uses an open source small model to correct the time parameter. Specifically, the open source small model is first used to classify user questions into three categories: 1) questions that do not contain time, 2) questions that contain time points, and 3) questions that contain time periods. Then, the open source small model is used to extract parameters based on the classification results. Parameter conversion is performed according to the application system interface standard requirements, such as converting March 2024 to 202403. The following is an example of user questions and time parameter extraction:

[0171] ----0----

[0172] query: Query the number of 5G base stations in Shanghai

[0173] Time category: No time

[0174] ----1----

[0175] query: Query the number of 5G base stations in Shanghai in May 2024

[0176] Time category: time point

[0177] Time:202405

[0178] ----2----

[0179] query: Query the number of 5G base stations in Shanghai in the past 10 months

[0180] Time category: Time period

[0181] Start time:202309

[0182] End time:202407

[0183] S4. Search and retrieve data based on the optimized Prompt to obtain corresponding target data.

[0184] In this step, data is retrieved and extracted from a data source based on the optimized Prompt, so that target data corresponding to the user's question can be obtained.

[0185] In some embodiments, an output post-processing step is also included, specifically as follows:

[0186] 1. Supplementary work parameters

[0187] Since there are fixed supplementary parameters, such as filed, method, and showWay, which cannot be obtained from user questions, these supplementary parameters are agreed upon according to the intent and application system interface input parameters. At this time, they need to be supplemented through post-processing according to the intent number.

[0188] 2. Parameter content correction

[0189] The large language model is a generative model, and the generated content has a certain degree of randomness. For example, the generated content sometimes contains free-to-use content such as carriage returns, spaces, and comments. For example, in some samples, although the generated content basically meets the needs, it still contains irrelevant content such as "backticks", "json", "carriage returns", and "spaces" that cannot be directly parsed. Post-processing is required to correct it and obtain standard format json parameters that can be processed by the backend.

[0190] 3. Parameter formatting

[0191] The output of the large language model is in string text format, and the output needs to be converted into JSON format so that it can be passed to the backend for further processing.

[0192] For example, the following is an example of json parameters after conversion, correction and post-processing by the small model. It can be seen that the values ​​corresponding to "province" and "indicator name" are corrected, and after correction by "small model 3", the relevant time parameters can be obtained accurately and stably. The format of the parameters is also standard json. The following is an example of user input and large language model output parameter results after optimizing Prompt and small model processing:

[0193] query: How has the number of 5G base stations in Shanghai changed in the past 10 months?

[0194] out:

[0195] {

[0196] "type":"BI"

[0197] "params":[

[0198] {

[0199] "Start time":"202309"

[0200] "End time":"202407"

[0201] "Province":"Shanghai"

[0202] "Indicator name": "Number of 5G medium-frequency base stations"

[0203] "field":"value"

[0204] "method":"query"

[0205] } ]

[0207] showway":"fold"

[0208] }

[0209] In some embodiments, it also includes: performing semantic splicing based on the target data obtained from the data to obtain an answer text corresponding to the user question.

[0210] Specifically, for data acquisition and result display, after post-processing to obtain json parameters, the backend obtains data through a specific application system interface, and performs semantic splicing to obtain the expected answer text and display content that the user wants. For example, a user asks: How many 5G base stations were there in Shanghai yesterday? The large language model correctly understands the user's intention, and parses the time, location, and indicator name before passing the json to the backend. The backend performs splicing based on the reply template. The reply template for this question is: s% (time) s% (location) s% (indicator name) is s% (value). The value returned through json and the interface replaces the placeholder to obtain the complete answer content and forward it to the front-end development.

[0211] After the backend sends the spliced ​​content to the frontend, the frontend displays the results in the APP Q&A window based on the extracted parameters and fixed supplementary parameters, such as text, pictures, jump links and other content. At this point, the entire intelligent BI data acquisition task from user questions to answers and displays is completed, and the process is accurate, fast and stable.

[0212] The present application proposes an intelligent BI data acquisition method, device and readable storage medium, which converts and corrects the indicator parameters, time parameters, address parameters, etc. extracted by the large language model through a small model to obtain the standard parameters required for data acquisition by the application system interface, thereby achieving fast and accurate intelligent BI data acquisition.

[0213] It should be understood that, although the various steps in the flowcharts in the above-described embodiments are sequentially displayed according to the indications of the arrows, these steps are not necessarily executed sequentially in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and they can be executed in other orders. Moreover, at least a portion of the steps in the figure may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily to be carried out sequentially, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0214] Figure 3 A schematic diagram of an intelligent BI data acquisition device provided in an embodiment of the present application, such as Figure 3 As shown, the present application provides an intelligent BI data acquisition device, the device comprising:

[0215] An intention determination module 11 is configured to obtain a user question and determine a target intention most relevant to the user question through a large language model;

[0216] A prompt determination module 12, which is configured to determine a corresponding task description prompt based on the target intention;

[0217] A parameter optimization module 13, which is configured to perform parameter optimization processing on the task description Prompt through a small model to obtain an optimized Prompt;

[0218] The search and data acquisition module 14 is configured to perform search and data acquisition based on the optimized Prompt to obtain corresponding target data.

[0219] Regarding the limitation of the intelligent BI data acquisition device, please refer to the limitation of the intelligent BI data acquisition method in the above embodiments of the present application, and this embodiment will not be repeated here.

[0220] Figure 4 Another schematic diagram of the intelligent BI data acquisition device provided in the embodiment of the present application is as follows Figure 4 As shown, in some embodiments, the present application provides an intelligent BI data acquisition device, including a memory 22 and a processor 21, the memory stores a computer program, and the processor is configured to run the computer program to execute the intelligent BI data acquisition method in the above-mentioned embodiments of the present application.

[0221] The memory is connected to the processor, the memory may be a flash memory or a read-only memory or other memory, and the processor may be a central processing unit or a single-chip microcomputer.

[0222] In some embodiments, the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the intelligent BI data acquisition method in the above-mentioned embodiments of the present application is implemented.

[0223] The computer-readable storage medium includes volatile or non-volatile, removable or non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, computer program modules or other data). Computer-readable storage media include, but are not limited to, RAM (Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable read only memory), flash memory or other memory technology, CD-ROM (Compact Disc Read-Only Memory), digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer.

[0224] It is to be understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of the present application, but the present application is not limited thereto. For those skilled in the art, various modifications and improvements can be made without departing from the spirit and substance of the present application, and these modifications and improvements are also considered to be within the scope of protection of the present application.

Claims

1. An intelligent BI data acquisition method, characterized in that: The method comprises: S1. Obtain user questions and determine the target intent most relevant to the user questions through a large language model; S2. Based on the target intention, determine the corresponding task description Prompt; S3, performing parameter optimization processing on the task description Prompt through the small model to obtain an optimized Prompt; S4. Search and retrieve data based on the optimized Prompt to obtain corresponding target data.

2. The intelligent BI data acquisition method according to claim 1, characterized in that: In S1, the target intent most relevant to the user question is determined through a large language model, including: Performing intent recognition on the user question through a large language model to obtain an intent recognition result; Filtering out a target intent most relevant to the user question from a plurality of preset intents according to the intent recognition result; Among them, the preset intention is pre-set based on the demand scenario.

3. The intelligent BI data acquisition method according to claim 2 is characterized in that: S2, including: Perform parameter parsing on the user question through a large language model to obtain a parameter parsing result; According to the parameter analysis result, the target task most relevant to the user question is selected from a plurality of preset tasks corresponding to the target intent, and a task description prompt corresponding to the target task is determined; Wherein, the preset task is pre-set based on the preset intention.

4. The intelligent BI data acquisition method according to claim 3 is characterized in that: S3, including: Based on the task description Prompt, extract parameters of the user question through a large language model to obtain indicator parameters; Using a small model, select the target parameter that is most similar to the indicator parameter from the standard parameter name library; The optimized prompt is obtained based on the target parameter that is most similar to the indicator parameter.

5. The intelligent BI data acquisition method according to claim 4 is characterized in that: The target parameters most similar to the indicator parameters are selected from the standard parameter name library through a small model, including: Using the NLTK edit distance similarity matching model, multiple standard parameters similar to the indicator parameters are selected from the standard parameter name library; Through a deep neural network similarity matching model, a target parameter most similar to the indicator parameter is selected from a plurality of standard parameters similar to the indicator parameter.

6. The intelligent BI data acquisition method according to claim 1, characterized in that: S3, also includes: According to the user's question, the prompt description of the time parameter extraction task is optimized, and the optimized time parameters are obtained based on the optimized prompt.

7. The intelligent BI data acquisition method according to any one of claims 1 to 6, characterized in that: Also includes: Semantic splicing is performed based on the target data obtained from the data collection to obtain the answer text corresponding to the user's question.

8. An intelligent BI data acquisition device, characterized in that: The device comprises: An intent determination module configured to obtain a user question and determine a target intent most relevant to the user question through a large language model; A prompt determination module, which is configured to determine a corresponding task description prompt based on the target intention; A parameter optimization module, which is configured to perform parameter optimization processing on the task description Prompt through a small model to obtain an optimized Prompt; The search and data acquisition module is configured to perform search and data acquisition based on the optimized Prompt to obtain corresponding target data.

9. An intelligent BI data acquisition device, characterized in that: It comprises a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to implement the intelligent BI data acquisition method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the intelligent BI data acquisition method according to any one of claims 1 to 7 is implemented.

Images