Data retrieval methods, devices, electronic equipment, storage media and software products

Abstract

This invention relates to the field of data processing technology, and discloses a data retrieval method, apparatus, electronic device, storage medium, and program product. The method includes: obtaining descriptive information corresponding to a data query question; retrieving, from a pre-built data knowledge base, a target field matching the query intent and the target data storage unit containing the target field, based on the query intent corresponding to the descriptive information; generating a target structured query statement corresponding to the data query question based on the target field, the target data storage unit, and the descriptive information; and retrieving the target data corresponding to the data query question from the data knowledge base using the target structured query statement. By implementing this technical solution, the differences between natural language and standard database storage are bridged from the source, avoiding deviations in the generation of structured query statements, enhancing the trustworthiness of the data retrieval process and results, and improving the accuracy of target data retrieval.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and more specifically to data retrieval methods, devices, electronic devices, storage media, and program products. Background Technology

[0002] As enterprises deepen their digital transformation, data has become a core production factor. Currently, Natural Language to Structured Query Language (NL2SQL) is commonly used to query databases and retrieve relevant data for analysis. However, NL2SQL struggles to dynamically perceive the differences between colloquial expressions and standard database storage. Lacking strong contextual constraints, it is prone to selecting incorrect fields to generate SQL, leading to biased data retrieval results and impacting the accuracy of data analysis. Summary of the Invention

[0003] This invention provides a data retrieval method, apparatus, electronic device, storage medium, and program product to solve the problem of inaccurate data retrieval results.

[0004] In a first aspect, the present invention provides a data retrieval method, comprising: obtaining descriptive information corresponding to a data query question; retrieving, from a pre-constructed data knowledge base, a target field matching the query intent and the target data storage unit where the target field is located, based on the query intent corresponding to the descriptive information, wherein the data knowledge base is constructed based on data mapping relationships, and the data mapping relationships are determined based on non-standard dimension information, standard dimension information, and data hierarchical path information; generating a target structured query statement corresponding to the data query question based on the target field, the target data storage unit, and the descriptive information; and retrieving target data corresponding to the data query question from the data knowledge base using the target structured query statement.

[0005] The data retrieval method provided in this invention constructs a data mapping relationship through non-standard dimensional information, standard dimensional information, and data hierarchical path information. This enables the unified mapping of non-standard expressions such as colloquialisms, abbreviations, and alternative names to standard dimensions, bridging the gap between natural language and standard database storage from the source. Based on the query intent corresponding to the descriptive information, a search is conducted in a pre-built data knowledge base to quickly locate the target field and its corresponding target data storage unit through the data mapping relationship. This effectively avoids field mismatch and dimension misjudgment, improving the accuracy of matching target fields with data storage units. Based on the target field and target data storage unit determined by the search, a structured query statement is generated. The generation boundary of the structured query statement is constrained by the target field and its corresponding target data storage unit, reducing illusions and avoiding deviations in the structured query statement caused by free generation. This ensures the generation of compliant and executable structured query statements, thereby guaranteeing the authenticity and reliability of the data retrieval results obtained based on the structured query statement, enhancing the trustworthiness of the data retrieval process and results, and improving the accuracy of target data retrieval.

[0006] In one optional implementation, based on the query intent corresponding to the description information, a target field matching the query intent and the target data storage unit where the target field is located are retrieved from a pre-built data knowledge base. This includes: extracting the query entity corresponding to the description information based on the query intent corresponding to the description information; and retrieving the data knowledge base based on the query entity to obtain the target field matching the query entity and the target data storage unit where the target field is located. The standard dimension information corresponding to the target field is the query entity.

[0007] The data retrieval method provided in this invention parses the description information by query intent, thereby removing redundant semantic information in the description information and focusing on the core query entity. This can significantly narrow the scope of data retrieval, avoid irrelevant information from interfering with the data matching process, ensure the uniqueness and accuracy of the matching of the target field and its target data storage unit, and effectively avoid matching deviations caused by fields with the same name in multiple tables or similar dimension values.

[0008] In one optional implementation, the data knowledge base is retrieved based on the query entity to obtain the target field matching the query entity and the target data storage unit where the target field is located. This includes: obtaining each dimension field of the data knowledge base, the standard field values ​​and non-standard field values ​​corresponding to each dimension field, and the entity alias corresponding to the query entity; performing fuzzy matching between the query entity and each standard field value to obtain a first field query result; matching the entity alias with the non-standard field value to obtain a second field query result; matching the query entity with each standard field value based on data hierarchy path information to obtain a third field query result; and determining the target field and the target data storage unit where the target field is located based on the fusion result of the first field query result, the second field query result, and the third field query result.

[0009] The data retrieval method provided in this invention improves the coverage and accuracy of dimensional field matching by performing fuzzy matching, alias matching and data path hierarchical matching in parallel, and integrates multi-source retrieval results, thereby avoiding matching blind spots and achieving disambiguation at the field level.

[0010] In one optional implementation, the target field and the target data storage unit where the target field is located are determined based on the fusion result of the query results of the first field, the query results of the second field, and the query results of the third field. This includes: obtaining at least one first candidate field and the first candidate data storage unit where each first candidate field is located based on the fusion result of the query results of the first field, the query results of the second field, and the query results of the third field; and performing disambiguation processing on each first candidate field and each first candidate data storage unit based on a preset disambiguation strategy to obtain the target field and the target data storage unit where the target field is located. The preset disambiguation strategy includes one or more of the following: a text similarity disambiguation strategy, a matching degree disambiguation strategy between dimension field hierarchy and descriptive information, and a historical query field disambiguation strategy.

[0011] The data retrieval method provided in this invention integrates multi-source retrieval outputs and introduces a multi-factor pre-defined disambiguation strategy to select the optimal target field and corresponding target data storage unit from a candidate field set. This effectively resolves semantic conflicts caused by fields with the same name or similar dimension values ​​across multiple tables, ensuring the uniqueness and reliability of field location. Furthermore, this field disambiguation process is fully automated without manual intervention, maintaining millisecond-level response times. This significantly improves the accuracy of field matching in complex semantic scenarios, providing a reliable and unique field basis for the subsequent generation of structured query statements.

[0012] In one optional implementation, determining the target field and the target data storage unit where the target field is located based on the fusion result of the query results of the first field, the second field, and the third field includes: obtaining at least one second candidate field and the second candidate data storage unit where each second candidate field is located based on the fusion result of the query results of the first field, the second field, and the third field; performing disambiguation processing on each second candidate field and each second candidate data storage unit to obtain at least one third candidate field and the third candidate data storage unit where each third candidate field is located; and determining the target field and the target data storage unit where the target field is located in response to the selection operation on the third candidate field and the third candidate data storage unit.

[0013] The data retrieval method provided in this invention, after multi-strategy retrieval fusion, significantly reduces the scope of manual intervention through automatic coarse screening. Manual intervention compensates for the limitations of automatic disambiguation in highly ambiguous scenarios, balancing retrieval efficiency and result reliability, and reducing matching errors in extremely ambiguous scenarios. This approach ensures rapid response for regular queries while mitigating the risk of misselected fields in extremely ambiguous scenarios, significantly improving the credibility of query results. It can adapt to various business scenarios with complex dimensional systems, diverse colloquial expressions, and overlapping business definitions.

[0014] In one optional implementation, a target structured query statement corresponding to a data query problem is generated based on the target field, the target data storage unit, and the descriptive information. This includes: performing structured processing on the target field and the target data storage unit based on a preset constraint template to obtain structured constraint information; determining query hints based on the structured constraint information and the descriptive information; and guiding the generation process of the structured query statement based on the query hints to obtain the target structured query statement corresponding to the data query problem.

[0015] The data retrieval method provided in this invention uses structured constraint information to lock the target field, target data storage unit, and dimension field values, accurately guiding the generation direction of query suggestions. It restricts the generation space of structured query statements to the scope of structured constraint information, effectively suppressing problems such as dimension field illusion, table name mismatch, and fictitious dimension field values. This ensures that the target structured query statement is syntactically compliant, logically rigorous, and directly executable, which helps to significantly improve the accuracy of data retrieval results.

[0016] In one optional implementation, determining query suggestions based on structured constraint information and description information includes: determining the query logic structure of the structured query statement based on the description information, and setting indicator fill positions in the query logic structure; generating explicit suggestions for the structured query statement based on the structured constraint information; filling the indicator fill positions with standard dimension information corresponding to the target field to obtain indicator aggregation suggestions; and concatenating the description information, explicit suggestions, and indicator aggregation suggestions to obtain query suggestions.

[0017] The data retrieval method provided in this invention determines the query logic structure through descriptive information, generates explicit prompts by combining structured constraint information, and fills the query logic structure with standard dimension information corresponding to the target field to obtain corresponding indicator aggregation prompts. The descriptive information, explicit prompts, and indicator aggregation prompts are concatenated to obtain query prompts with clear structure and well-defined constraints, providing precise guidance for the subsequent generation of structured query statements. Furthermore, by using explicit prompts and structured constraint information, misunderstandings caused by mixed information are avoided, enabling accurate adaptation of data queries in zero-sample or low-sample scenarios. This effectively guides the generation process of structured query statements, improves the accuracy and reliability of structured query statement generation, and facilitates adaptation to multi-dimensional data query needs.

[0018] In an optional implementation, the method further includes: validating the target structured query statement to obtain a statement validation result; if the validation result indicates that the target structured query statement has passed validation, then performing the step of retrieving the target data corresponding to the data query question from the data knowledge base using the target structured query statement; if the validation result indicates that the target structured query statement has failed validation, then generating an error reminder message.

[0019] The data retrieval method provided in this invention verifies the target structured query statement to intercept invalid or erroneous structured query statements before data retrieval is performed, thus preventing invalid queries from consuming retrieval resources and causing database retrieval anomalies. Simultaneously, it provides accurate error alerts when verification fails, facilitating rapid identification of errors so users can make accurate corrections. This allows the system to automatically learn and correct content, achieving self-learning in the generation of structured query statements.

[0020] In one optional implementation, a data knowledge base is constructed based on data mapping relationships, including: obtaining multiple dimension fields in each data storage unit; determining standard dimension information, non-standard dimension information, and data hierarchy path information corresponding to each dimension field; constructing data mapping relationships based on data hierarchy path information, standard dimension information, and non-standard dimension information; constructing an inverted index for each dimension field based on the data mapping relationships; and obtaining the data knowledge base based on the inverted indexes of each dimension field.

[0021] The data retrieval method provided in this invention, when constructing a data knowledge base, extracts all dimension fields to build structured knowledge containing standard and non-standard dimension information and data hierarchy path information, rather than simply storing single dimension values. Therefore, this data knowledge base can effectively solve the challenges of synonym mapping and field selection under multiple table structures, providing a solid, reliable, and scalable data foundation for subsequent data retrieval. By constructing data mapping relationships and unifying the definition of dimension fields, matching obstacles caused by colloquial biases are eliminated from the ground up. By constructing an inverted index, retrieval response time is significantly shortened, balancing dimension field matching coverage with query real-time performance.

[0022] In an alternative implementation, the method further includes: updating the data mapping relationship of the dimension field in response to a configuration operation for non-standard dimension information, and / or in response to a change in the dimension field; and updating the data knowledge base based on the updated data mapping relationship.

[0023] The data retrieval method provided in this invention ensures that the data knowledge base remains consistent with the actual data scenario by real-time monitoring of information changes corresponding to dimension fields, adapting to dynamic needs such as data business iteration, dimension adjustments, and alias additions. Simultaneously, it dynamically updates the data knowledge base using incremental updates, avoiding the resource consumption caused by a full reconstruction of the data knowledge base, improving the update efficiency of the data knowledge base, and achieving self-evolution of the data knowledge base.

[0024] Secondly, the present invention provides a data retrieval device, comprising: an acquisition module for acquiring descriptive information corresponding to a data query question; a retrieval module for retrieving, based on the query intent corresponding to the descriptive information, a target field matching the query intent and the target data storage unit where the target field is located from a pre-built data knowledge base, wherein the data knowledge base is constructed based on data mapping relationships, and the data mapping relationships are determined based on non-standard dimension information, standard dimension information, and data hierarchical path information; a query statement generation module for generating a target structured query statement corresponding to the data query question based on the target field, the target data storage unit, and the descriptive information; and a data retrieval module for retrieving target data corresponding to the data query question from the data knowledge base using the target structured query statement.

[0025] Thirdly, the present invention provides an electronic device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the data retrieval method described in the first aspect or any corresponding embodiment thereof.

[0026] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to perform the data retrieval method described in the first aspect or any corresponding embodiment thereof.

[0027] Fifthly, the present invention provides a computer program product, including computer instructions for causing a computer to execute the data retrieval method described in the first aspect or any corresponding embodiment thereof. Attached Figure Description

[0028] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0029] Figure 1 This is a schematic diagram of an application scenario according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the first type of data retrieval method according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the construction process of a data knowledge base according to an embodiment of the present invention; Figure 4 This is a schematic diagram of a second process for a data retrieval method according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the third process of the data retrieval method according to an embodiment of the present invention; Figure 6 This is a structural block diagram of a data retrieval device according to an embodiment of the present invention; Figure 7 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in the present invention and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.

[0032] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0033] Current NL2SQL has significant shortcomings when dealing with complex business scenarios, mainly in the following two aspects: (1) Semantic gap of dimensional values: There is a huge difference between users’ colloquial expressions (such as “mathematics college”) and standard stored values ​​in the data warehouse (such as “mathematics college”). Existing large language models (LLM) rely on pre-trained prior knowledge and lack the ability to dynamically perceive specific private business dictionaries, which can easily lead to entity recognition failure.

[0034] (2) Ambiguous positioning of dimension fields: The same business concept often corresponds to multiple physical fields (such as "first-level department" and "second-level department", or the same dimension in different tables). The existing Large Language Model (LLM) lacks strong context constraints, making it easy to select the wrong field to generate SQL, resulting in deviation of query results.

[0035] To address the aforementioned issues, current methods primarily rely on LLM's contextual understanding or vector similarity retrieval, which struggles to achieve high-precision "value normalization" and "field-level disambiguation" while ensuring real-time performance.

[0036] Based on this, the technical solution of this invention constructs a data mapping relationship based on non-standard dimensional information, standard dimensional information, and data hierarchy path information. It then builds a data knowledge base based on this data mapping relationship to bridge the gap between the standards of natural language and standard database storage. Subsequently, relying on the pre-built data knowledge base, it performs precise matching of fields and dimension values ​​to pinpoint the target field and its corresponding target data storage unit during the retrieval phase. This avoids confusion between multiple tables, ensures that the generated SQL strictly correlates the matching results of the target field and the target data storage unit, reduces SQL generation bias, and thus avoids data retrieval bias, improving data retrieval accuracy.

[0037] As an optional application scenario of this invention, such as Figure 1As shown, the application scenario includes at least one electronic device 110 equipped with an intelligent data query system 101, and a user 130 can interact with the intelligent data query system 101 through the electronic device 110 and / or the access device of the electronic device 110.

[0038] For example, the intelligent question-and-answer system 101 can be an application that provides data query and question-and-answer related services. Figure 1 In the application scenario shown, if the intelligent data query system 101 is active, the electronic device 110 can display the front-end interface 102 of the intelligent data query system 101. This front-end interface 102 can include various pages provided by the intelligent data query system 101, such as interactive pages, settings pages, query pages, etc.

[0039] In some embodiments, electronic device 110 is communicatively connected to server 120 to provide services to intelligent query system 101. Electronic device 110 may be a mobile terminal, fixed terminal, or portable terminal, etc., including but not limited to mobile phones, desktop computers, laptop computers, multimedia tablets, e-book devices, gaming devices, or any combination thereof, including accessories and peripherals of these devices, or any combination thereof. In some embodiments, electronic device 110 may also support any type of interface, and server 120 may be various types of computing systems or servers capable of providing computing power, including but not limited to mainframes, edge computing nodes, computing devices in cloud environments, etc.

[0040] It should be noted that, Figure 1 This is merely an example of an application scenario and does not limit the scope of protection of this invention.

[0041] The embodiments of the present invention will now be described with reference to the accompanying drawings. It should be understood that the pages shown in the drawings are merely examples, and various page designs are possible in practice. The various graphic elements on the page may have different arrangements and different visual representations; one or more elements may be omitted or replaced, and one or more other elements may also be present, without any limitation in the embodiments of the present invention. Furthermore, the embodiments described below primarily pertain to electronic device 110. It should be understood that the actions described relative to electronic device 110 can be performed by the intelligent query system 101 on electronic device 110, or can be performed by the intelligent query system 101 in conjunction with its server (e.g., server 120).

[0042] According to an embodiment of the present invention, a data retrieval method embodiment is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0043] This embodiment provides a data retrieval method that can be used in the aforementioned electronic devices, such as computers and tablets. The electronic devices are equipped with a Natural Language Query for BI system driven by a large language model. This system acts as an intermediary layer connecting users and the data warehouse, and is used to convert users' natural language into structured query language.

[0044] Figure 2 This is a flowchart of a data retrieval method according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps: Step S201: Obtain the description information corresponding to the data query question.

[0045] The data query question is the question the user currently needs to query; the description information is a description of the data query question using natural language.

[0046] Specifically, the intelligent query system provides a front-end interactive interface. Users can input data to the intelligent query system in the form of natural language to query questions, and the text in the form of natural language will then serve as the corresponding descriptive information.

[0047] For example, if you enter "Statistics on the number of students in the School of Mathematics in XX year" in the front-end interactive interface, then the complete question will be the description information corresponding to this data query question.

[0048] Of course, users can also describe their data query questions in voice. The intelligent data query system can convert the voice description of the data query question into a corresponding text description through voice-to-text conversion, and display the text description on the front-end interactive interface so that users can confirm whether the description information is accurate.

[0049] Step S202: Based on the query intent corresponding to the description information, retrieve the target field that matches the query intent and the target data storage unit where the target field is located from the pre-built data knowledge base.

[0050] The data knowledge base is constructed based on data mapping relationships, which are determined based on non-standard dimensional information, standard dimensional information, and data hierarchical path information.

[0051] The query intent represents the data query purpose identified from the description information; the data mapping relationship is the association and correspondence between non-standard dimension information, standard dimension information, and data hierarchy path information; non-standard dimension information is the dimension description in user colloquial, abbreviated, alternative, or misspelled form; standard dimension information is the standardized value stored in the dimension field in the database; data hierarchy path information is the sequence of hierarchical relationships between dimension values; data storage unit represents an independent table, partitioned table, or data view in the database; and the data knowledge base is a structured data collection that pre-builds and stores dimension mapping relationships and indexes.

[0052] The intelligent query system uses a large language model to parse the descriptive information and determine the query intent corresponding to the current descriptive information. Based on the parsed query intent, it determines the key information to be retrieved. Then, it queries the data knowledge base with the key information corresponding to the query intent, matches the target field associated with it, and determines the target data storage unit for storing the target field.

[0053] Continuing with the previous example, by parsing the description information, we can determine that the query intent is "number of students in the School of Mathematics". At this point, in the pre-built data knowledge base, data retrieval according to the query intent can match the corresponding dimension field "School Name", which is the target field. If this target field is stored in the business dimension table "dim_department", then that business dimension table "dim_department" is the corresponding target data storage unit.

[0054] Step S203: Based on the target field, target data storage unit, and description information, generate a target structured query statement corresponding to the data query problem.

[0055] The system determines the data to be queried based on the description information; it also determines the location of the data to be queried based on the retrieved target fields and the target data storage unit where the target fields are located. The retrieved target fields and target data storage units are integrated, and combined with the key query semantics carried in the description information, the system is guided to generate a structured query statement, resulting in a grammatically correct and directly executable structured query statement (SQL) that can be directly parsed and executed by the database.

[0056] Using the previous example, combining the target field "College Name", the target data storage unit "dim_department", and the time and indicator information in the description, generate a compliant SQL statement: SELECT student_num FROM dim_department WHERE college_name='College of Mathematics' AND year=XX.

[0057] Step S204: Use the target structured query statement to retrieve the target data corresponding to the data query question from the data knowledge base.

[0058] The intelligent query system submits the generated target structured query statement to the corresponding database engine, which then parses and executes the target structured query statement to extract the target data that matches the conditions from the target data storage unit.

[0059] After the target data is organized, the organized target data is fed back to the front-end interactive interface of the intelligent data analysis system for users to view.

[0060] Following the previous example, execute the data query according to the generated target structured query statement, and return the numerical result of the number of students in the School of Mathematics in XX year, thus completing this data query.

[0061] The data retrieval method provided in this embodiment constructs a data mapping relationship through non-standard dimensional information, standard dimensional information, and data hierarchical path information. This enables the unified mapping of non-standard expressions such as colloquialisms, abbreviations, and alternative names to standard dimensions, bridging the gap between natural language and standard database storage from the source. Based on the query intent corresponding to the descriptive information, a search is conducted in a pre-built data knowledge base to quickly locate the target field and its corresponding target data storage unit through the data mapping relationship. This effectively avoids field mismatch and dimension misjudgment, improving the accuracy of matching target fields and data storage units. Based on the target field and target data storage unit determined by the search, a structured query statement is generated. The generation boundary of the structured query statement is constrained by the target field and its corresponding target data storage unit, reducing illusions and avoiding deviations in the structured query statement caused by free generation. This ensures the generation of compliant and executable structured query statements, thereby guaranteeing the authenticity and reliability of the data retrieval results obtained based on the structured query statement and enhancing the trustworthiness of the data retrieval process and results.

[0062] In data environments, inconsistent naming conventions, complex aliases, and scattered hierarchical relationships among dimension fields directly impact the matching accuracy and stability of natural language queries. This embodiment provides a method for constructing a data knowledge base based on data mapping relationships. Through systematic extraction, standardized definition, structured mapping, and indexing, the data knowledge base is constructed, providing reliable underlying support for subsequent data retrieval.

[0063] Figure 3 This is a flowchart illustrating the construction of a data knowledge base based on data mapping relationships according to an embodiment of the present invention, such as... Figure 3 As shown, the process includes the following steps: Step S301: Obtain multiple dimension fields from each data storage unit.

[0064] Periodically iterate through all data storage units (i.e., database tables, partitioned tables, or data views) within the database, filter and extract each dimension field and all its unique values ​​in each data storage unit.

[0065] Using the above example of querying the number of students, we iterate through the college dimension table, college budget dimension table, and college major dimension table in the school database, and extract dimension fields such as college name, budget item name, and major name.

[0066] Step S302: Determine the standard dimension information, non-standard dimension information, and data hierarchy path information corresponding to each dimension field.

[0067] As described above, standard dimension information refers to the standardized values ​​of dimension fields; non-standard dimension information refers to dimension values ​​in colloquial, abbreviated, alternative, or misspelled forms; data hierarchy path information represents the hierarchical relationship of data from top to bottom, such as School A → College B → Major C, etc.

[0068] For each dimension field, the system sequentially queries its corresponding standard dimension information, non-standard dimension information, and data hierarchy path information. Standard dimension information can include: the schema and table name (schema_table) to which the dimension field belongs (used for precise context filtering), the physical field name (field_name), and the standard dimension value (canonical_value). Non-standard dimension information can include a set of aliases, which can be pre-configured manually or automatically mined by the intelligent query system based on the user's historical queries; these aliases can include pinyin, abbreviations, homophones, and simplified characters. Data hierarchy path information is used to assist in disambiguation during field queries, ensuring accurate location of the data storage unit and avoiding misjudgments due to the same field residing in multiple data storage units.

[0069] Of course, you can also obtain occurrence_stats, which are associated with the dimension fields, to quantify the distribution characteristics and usage popularity of the dimension field values, which is convenient for subsequent weighting and sorting.

[0070] Step S303: Construct a data mapping relationship based on data hierarchy path information, standard dimension information and non-standard dimension information.

[0071] Data hierarchical path information, standard dimension information, and non-standard dimension information are associated and bound to establish a one-to-one structured mapping, forming corresponding data mapping relationships, such as data hierarchical path information – standard dimension information – non-standard dimension information.

[0072] Using the "College Name" field as an example, we can construct a data mapping relationship between the College of Mathematics and its aliases: School → College of Mathematics → such as College of Mathematics, School of Mathematics, SXY, etc.

[0073] Step S304: Based on the data mapping relationship, construct an inverted index for each dimension field.

[0074] An inverted index represents an index structure that uses keywords as index terms, pointing to corresponding dimension fields and mapped records. Specifically, for each dimension field, an inverted index is built using keywords from both standard and non-standard dimension information in the data mapping relationship as retrieval entry points, combined with data hierarchy path information.

[0075] For example, keywords such as "School of Mathematics" and "School of Economics and Management" can be mapped to the corresponding records in the "School Name" field, enabling quick location of keywords to dimension fields.

[0076] Step S305: Based on the inverted index of each dimension field, obtain the data knowledge base.

[0077] All dimension fields, data mapping relationships, and corresponding inverted indexes are organized into a structured, searchable, and maintainable data collection, i.e., a data knowledge base. This data knowledge base centrally carries dimension definitions, aliasing systems, and hierarchical structures, becoming the core basis for dimension matching and field location in natural language querying.

[0078] After the data knowledge base is built, it is written into the full-text search engine of the intelligent data retrieval system to support subsequent data retrieval. After being written into the full-text search engine, the data knowledge base supports multi-tenant isolation to ensure that the dimensional knowledge, mapping relationships and retrieval resources of different tenants are isolated from each other, without interference or leakage.

[0079] In some alternative implementations, in order to adapt to dynamic scenarios such as data changes, dimension adjustments, and the addition of aliases, and to continuously maintain the accuracy of search matching, it is necessary to incrementally update the data knowledge base to ensure that the data knowledge base is synchronized with the actual data.

[0080] Accordingly, the methods for constructing the aforementioned data knowledge base also include: Step a1: In response to configuration operations for non-standard dimension information, and / or, in response to changes in dimension fields, update the data mapping relationship of the dimension fields.

[0081] Step a2: Update the data knowledge base based on the updated data mapping relationship.

[0082] The configuration operation can detect the non-standard dimension information corresponding to each dimension field in the database in real time. This configuration operation can include adding, modifying, and deleting non-standard dimension values ​​(aliases). For example, add "School of Mathematics" as an alias for "School of Mathematics".

[0083] At the same time, it can also monitor events such as the addition, deletion, renaming or change of dimension fields in the database in real time. For example, adding the dimension field "student origin" and deleting the dimension value "gender".

[0084] When the aforementioned changes are detected in a dimension field, the data mapping relationship between the "data hierarchy path information - standard dimension information - non-standard dimension information" of the corresponding dimension field is updated synchronously, and the inverted index of the corresponding dimension field is synchronized according to the updated data mapping relationship to dynamically update the data knowledge base.

[0085] In the above implementation, by monitoring changes in the information corresponding to the dimension fields in real time, the data knowledge base is ensured to always remain consistent with the actual data scenario, adapting to the dynamic needs of data business iteration, dimension adjustment, and alias addition. Simultaneously, incremental updates are used to dynamically update the data knowledge base, avoiding the resource consumption caused by a full reconstruction of the data knowledge base and improving the update efficiency.

[0086] The data knowledge base construction method provided in this embodiment extracts all dimension fields to build structured knowledge containing standard and non-standard dimension information and data hierarchy path information, rather than simply storing single dimension values. Therefore, this data knowledge base can effectively solve the challenges of synonym mapping and field selection under multiple table structures, providing a solid, reliable, and scalable data foundation for subsequent data retrieval. By constructing data mapping relationships and unifying the definition of dimension fields, matching obstacles caused by colloquial biases are eliminated from the bottom up. By constructing an inverted index, retrieval response time is significantly shortened, balancing dimension field matching coverage with query real-time performance.

[0087] This embodiment provides a data retrieval method that can be used in the aforementioned electronic devices, such as computers and tablets. The electronic devices are equipped with a Natural Language Query for BI system driven by a large language model. This system acts as an intermediary layer connecting users and the data warehouse, and is used to convert users' natural language into structured query language.

[0088] Figure 4 This is a flowchart of a data retrieval method according to an embodiment of the present invention, such as... Figure 4 As shown, the process includes the following steps: Step S401: Obtain the description information corresponding to the data query problem. For details, please refer to the relevant descriptions of the corresponding steps in the above embodiments, which will not be repeated here.

[0089] Step S402: Based on the query intent corresponding to the description information, retrieve the target field that matches the query intent and the target data storage unit where the target field is located from the pre-built data knowledge base.

[0090] The data knowledge base is constructed based on data mapping relationships, which are determined based on non-standard dimensional information, standard dimensional information, and data hierarchical path information.

[0091] Specifically, step S402 includes: Step S4021: Based on the query intent corresponding to the description information, extract the query entity corresponding to the description information.

[0092] A query entity is a key semantic unit within the descriptive information; that is, it is the concrete expression of the query request at the dimensional level. Specifically, after the intelligent query system receives the descriptive information input by the user, it performs semantic understanding of the descriptive information based on a large language model to determine the user's query intent. Based on the query intent, irrelevant words (such as adjectives and conjunctions) in the descriptive information are removed, and key semantic units that carry the core query object, dimensional range, or limiting conditions are identified and extracted. These key semantic units are the query entities.

[0093] In a specific example, if the description is "Please help me check the sales situation of all stores in Y area in month M", the intelligent query system uses a large language model to determine that the query intent is "to count store sales". Based on this query intent, the system extracts the query entities "month M", "Y area" and "stores" from the description.

[0094] Step S4022: Retrieve the data knowledge base based on the query entity to obtain the target field that matches the query entity and the target data storage unit where the target field is located.

[0095] The standard dimension information corresponding to the target field is the query entity.

[0096] Using the extracted query entities as search keywords, semantic matching is performed between them and the standard dimension information, non-standard dimension information, and data hierarchical path information corresponding to each dimension field in the data knowledge base to determine the semantic matching degree. The dimension field with the highest semantic matching degree is selected as the target field. At the same time, the target data storage unit where the target field is located is located by combining the data hierarchical path information.

[0097] Using the previous example, the extracted query entities "M month", "Y district", and "store" are input into the data knowledge base for retrieval. The target fields "region", "store name", and "time" are matched, and all three are stored in the business dimension table "dim_retail_store". This business dimension table "dim_retail_store" is the target data storage unit. Furthermore, the standard dimension information corresponding to the target field "region" includes "Y district", the standard dimension information corresponding to "store name" includes the standard names of each store, and the standard dimension information corresponding to "time" includes "M month", which is precisely aligned with the query entities.

[0098] In some optional implementations, step S4022 above includes: Step b1: Obtain the data knowledge base's various dimension fields, the standard field values ​​corresponding to each dimension field, and the entity aliases corresponding to the queried entity.

[0099] Read all dimension fields and their corresponding standard field values ​​(i.e., standard dimension values ​​stored in the database with consistent definitions) from the pre-built data knowledge base, and automatically retrieve or mine the corresponding entity aliases (i.e., non-standard expressions such as colloquialisms, abbreviations, and alternative names) in conjunction with the query entity.

[0100] For example, if the query entity is "R&D Department", the dimension fields obtained from the data knowledge base include "department name"; the corresponding standard field value includes "R&D Department"; the entity aliases corresponding to the query entity can include "development department", "research department", "technology department", etc.

[0101] Step b2 involves performing a fuzzy match between the query entity and the values ​​of each standard field to obtain the query result for the first field.

[0102] Edit distance is used to perform fuzzy matching between the query entity and the values ​​of various standard fields in the data knowledge base to solve problems such as colloquial wording deviations and approximate expressions. For example, if the query entity "R&D Department" is successfully fuzzily matched with the standard field values ​​"R&D Department, R&D Department, Technology R&D Department", the above standard field values ​​and their respective field dimensions are output to form the first field query result.

[0103] Step b3: Match the entity alias with the non-standard field value to obtain the query result for the second field.

[0104] As described above, non-standard dimension field information includes aliases corresponding to the dimension fields. Here, the entity alias corresponding to the query entity can be used as a search keyword to match the non-standard field values ​​corresponding to each dimension field, thus aligning the aliases. Alias ​​matching covers custom abbreviations, industry slang, or internal terminology, outputting the corresponding second field query results.

[0105] For example, the entity aliases "Development Department, Research Department" are matched with the alias "Research and Development Department" through data mapping relationship, and the corresponding dimension field "Department Name" of "Research and Development Department" is output, which constitutes the second field query result.

[0106] Step b4: Based on the data hierarchy path information, match the query entity with the values ​​of each standard field to obtain the query result of the third field.

[0107] Read the data hierarchy path information corresponding to the dimension field (i.e., the hierarchical relationship of the dimension field from top to bottom), combine the data hierarchy path information to perform context matching on the query entity, prioritize the standard field value that matches the data hierarchy path information, and use the dimension field corresponding to the standard field value as the third field query result.

[0108] For example, querying the entity "R&D Department" matches the data hierarchy path information "Group → Technology Business Unit → R&D Department", locates the standard field value "R&D Department" and the corresponding dimension field, and constitutes the third field query result.

[0109] Step b5: Based on the fusion result of the query results of the first field, the second field, and the third field, determine the target field and the target data storage unit where the target field is located.

[0110] The query results of the first field, the second field, and the third field are merged, deduplicated, and a candidate field set is formed. The consistency and matching relevance of the dimension fields are combined for comprehensive screening to determine the target field that best matches the query entity and its target data storage unit.

[0111] For example, the fusion result points to the dimension field "Department Name" and the standard field value "R&D Department", both of which originate from the same dimension table "dim_org". Therefore, "Department Name" is determined as the target field and "dim_org" is determined as the target data storage unit.

[0112] In the above implementation, by performing fuzzy matching, alias matching, and data path hierarchical matching in parallel, and by integrating multi-source retrieval results, the coverage and accuracy of dimensional field matching are improved, matching blind spots are avoided, and disambiguation at the field level is achieved.

[0113] In some alternative implementations, step b5 above may include: Step b51: Based on the fusion result of the query results of the first field, the second field, and the third field, at least one first candidate field and the first candidate data storage unit where each first candidate field is located are obtained.

[0114] Step b52: Based on the preset disambiguation strategy, disambiguation processing is performed on each first candidate field and each first candidate data storage unit to obtain the target field and the target data storage unit where the target field is located.

[0115] The query results of the three types of fields obtained by the above three types of field matching based on fuzzy matching, alias matching and data path hierarchical matching are summarized, duplicate records are removed, and matching items with similar semantics but different sources are merged to form a candidate field set. The candidate field set includes one or more first candidate fields, and each first candidate field is matched with its corresponding first candidate data storage unit.

[0116] Next, a preset disambiguation strategy is invoked to read the standard and non-standard dimension information corresponding to each first candidate field in the candidate field set from the data knowledge base. Multi-dimensional validation and priority sorting are performed on each first candidate field to verify whether each first candidate field has a unique mapping relationship with the query entity. Based on the validation results, the most matching first candidate field is determined as the target field from among multiple first candidate fields. Simultaneously, the legality of the attribution between the target field and its corresponding first candidate data storage unit is verified to confirm whether the correspondence between the target field and its corresponding first candidate data storage unit is unique. Therefore, based on the result of this attribution legality verification, the first candidate data storage unit containing the target field is determined as the target data storage unit.

[0117] The preset disambiguation strategies include one or more of the following: text similarity disambiguation strategy, matching degree disambiguation strategy between dimension field hierarchy and descriptive information, and historical query field disambiguation strategy.

[0118] Specifically, the text similarity disambiguation strategy disambiguates based on the degree of character matching between the query entity and standard or non-standard field values; the dimension field level and description information matching disambiguation strategy disambiguates based on the semantic fit between the data path level corresponding to the first candidate field and the query context corresponding to the description information; and the historical query field disambiguation strategy disambiguates based on the preference for frequently used fields in the past under the global dimension.

[0119] For example, if the query entity is "stores in City A", the fusion result includes the first candidate fields "store address" (from the store dimension table dim_store) and "city name" (from the city dimension table dim_city), which constitute two sets of first candidate fields and their corresponding first candidate data storage units.

[0120] In terms of text similarity, "City A" and "Store Address" are more closely related; in terms of dimensional field hierarchy matching, "Store Address" better fits the query context of "Store"; if store-related fields are frequently used in historical queries, their priority is further increased. After comprehensive multi-factor weighting and sorting, the optimal item "Store Address" is selected as the target field, and its corresponding target data storage unit is determined to be dim_store.

[0121] By integrating multi-source retrieval outputs and introducing a pre-defined disambiguation strategy with multiple factors, the optimal target field and its corresponding target data storage unit are selected from the candidate field set. This effectively resolves semantic conflicts caused by fields with the same name or similar dimension values ​​across multiple tables, ensuring the uniqueness and reliability of field location. Moreover, this field disambiguation process is fully automated without manual intervention, maintaining millisecond-level response times. This significantly improves the accuracy of field matching in complex semantic scenarios, providing a reliable and unique field basis for the subsequent generation of structured query statements.

[0122] In some alternative implementations, step b5 above may include: Step b51: Based on the fusion result of the query results of the first field, the second field, and the third field, at least one second candidate field and the second candidate data storage unit where each second candidate field is located are obtained. Step b52: Disambiguation processing is performed on each second candidate field and each second candidate data storage unit to obtain at least one third candidate field and the third candidate data storage unit where each third candidate field is located. Step b53, in response to the selection operation of the third candidate field and the third candidate data storage unit, determines the target field and the target data storage unit in which the target field is located.

[0123] The query results of the three types of fields obtained by the above three types of field matching based on fuzzy matching, alias matching and data path hierarchical matching are summarized, duplicate records are removed, and matching items with similar semantics but different sources are merged to form a candidate field set. The candidate field set includes one or more second candidate fields. Accordingly, each second candidate field matches its corresponding second candidate data storage unit.

[0124] Based on features such as text similarity, hierarchical matching degree of dimension fields, and frequency of occurrence statistics, multiple second candidate fields are sorted and filtered, and one or more third candidate fields with high matching degree and strong correlation and their corresponding third candidate data storage units are retained.

[0125] The filtered third candidate field and its corresponding third candidate data storage unit are presented in a visual format on the front-end interactive interface of the intelligent question-and-answer system, prompting the user to confirm. Once the user's selection is detected, the selected third candidate field and its corresponding third candidate data storage unit are locked and designated as the target field and target data storage unit.

[0126] After integrating multi-strategy retrieval, automatic coarse screening significantly reduces the scope of manual intervention. Manual intervention compensates for the limitations of automatic disambiguation in highly ambiguous scenarios, balancing retrieval efficiency and result reliability, and reducing matching errors in extremely ambiguous scenarios. This approach ensures rapid response for regular queries while mitigating the risk of misselected fields in extremely ambiguous scenarios, significantly improving the credibility of query results. It can adapt to various business scenarios with complex dimensional systems, diverse colloquial expressions, and overlapping business definitions.

[0127] Step S403: Based on the target field, target data storage unit, and description information, generate a target structured query statement corresponding to the data query question. For details, please refer to the relevant descriptions of the corresponding steps in the above embodiments; they will not be repeated here.

[0128] Step S404: Using the target structured query statement, retrieve the target data corresponding to the data query question from the data knowledge base. For details, please refer to the relevant descriptions of the steps in the above embodiments; they will not be repeated here.

[0129] The data retrieval method provided in this embodiment parses the description information through query intent to remove redundant semantic information in the description information and focus on the core query entity. This can significantly narrow the data retrieval scope, avoid irrelevant information from interfering with the data matching process, ensure the uniqueness and accuracy of the matching of the target field and its target data storage unit, and effectively avoid matching deviations caused by fields with the same name in multiple tables or similar dimension values.

[0130] This embodiment provides a data retrieval method that can be used in the aforementioned electronic devices, such as computers and tablets. The electronic devices are equipped with a Natural Language Query for BI system driven by a large language model. This system acts as an intermediary layer connecting users and the data warehouse, and is used to convert users' natural language into structured query language.

[0131] Figure 5This is a flowchart of a data retrieval method according to an embodiment of the present invention, such as... Figure 5 As shown, the process includes the following steps: Step S501: Obtain the description information corresponding to the data query problem. For details, please refer to the relevant descriptions of the corresponding steps in the above embodiments, which will not be repeated here.

[0132] Step S502: Based on the query intent corresponding to the description information, retrieve the target field that matches the query intent and the target data storage unit where the target field is located from the pre-built data knowledge base.

[0133] The data knowledge base is constructed based on data mapping relationships, which are determined based on non-standard dimensional information, standard dimensional information, and data hierarchical path information.

[0134] For details, please refer to the relevant descriptions of the steps in the embodiments shown above, which will not be repeated here.

[0135] Step S503: Based on the target field, target data storage unit, and description information, generate a target structured query statement corresponding to the data query problem.

[0136] Specifically, step S503 includes: Step S5031: Based on the preset constraint template, the target field and the target data storage unit are processed in a structured manner to obtain structured constraint information.

[0137] The default constraint template is a pre-defined template for structured processing, such as { "dim_field": "field_name", "allowed_values": ["standard_val1", "standard_val2"]}. Here, "dim_field" constrains the selection range of candidate fields, and "allowed_values" constrains the selection list of standard values.

[0138] The system invokes a pre-defined constraint template to encapsulate the retrieved target fields, their corresponding standard dimension values, and the target data storage unit into structured constraint information that is formatted correctly and can be parsed by a machine. This structured constraint information is used to limit the source of dimension fields, table affiliation, and the range of dimension values, preventing the model from acting arbitrarily.

[0139] For example, if the target field is "department name", the target data storage unit is dim_org, and the standard dimension value is "R&D Department", the structured constraint information obtained after processing by the preset constraint template is: {"target_table":"dim_org","target_field":"department name","allowed_values":["R&D Department"]}.

[0140] Step S5032: Determine query suggestion words based on structured constraint information and description information.

[0141] By integrating structured constraint information with the descriptive information corresponding to the data query question, semantically clear and well-defined query suggestions are generated. For example, structured constraint information is appended to the descriptive information to form query suggestions. These query suggestions, combined with query intent, hard constraints on fields and tables, and dimension value restrictions, provide hard constraints for the generation of structured query statements.

[0142] Continuing with the previous example, if the description is "count the current number of employees in the R&D department", combined with the structured constraints: {"target_table":"dim_org","target_field":"department name","allowed_values":["R&D department"]}, the query suggestion is generated as: count the current number of employees in the R&D department. To query the current number of employees in the R&D department, the query must use the table dim_org, the field department name, and the dimension values ​​must be limited to R&D department, generating a directly executable SQL statement.

[0143] In some optional implementations, step S5032 above includes: Step c1: Based on the description information, determine the query logic structure of the structured query statement, and set the indicator fill bits in the query logic structure.

[0144] Step c2: Based on the structured constraint information, generate explicit prompts for the structured query statement.

[0145] Step c3: Fill the indicator fill position with the standard dimension information corresponding to the target field to obtain the indicator aggregation prompt words.

[0146] Step c4: Combine the description information, explicit suggestion words, and aggregated indicator suggestion words to obtain the query suggestion words.

[0147] The query logic structure represents the core framework of a structured query statement; the indicator filler positions represent empty spaces used to hold statistical indicators or aggregate functions.

[0148] Specifically, the descriptive information based on natural language is parsed to extract the core framework of the structured query statement, namely the query logic structure; and index filling positions are reserved in the query logic structure to carry statistical indicators or aggregate functions.

[0149] For example, if the description is "Query the monthly sales of stores in City B in year Z", the query logic structure can be determined by parsing the description: SELECT [index fill position] FROM store dimension table WHERE city='B' AND year=ZGROUP BY month, where [index fill position] is a reserved empty space.

[0150] Extract hard constraints such as the target table, target field, and allowed dimension values ​​from the structured constraint information to generate explicit hints, which are used to enforce the scope of the generated structured query statement. For example, if the structured constraint information is {"target_table":"dim_store","target_field":"store city","allowed_values":["B "]}, the corresponding explicit hints generated are: the query table is limited to dim_store, the field is limited to store city, and the dimension value can only be B.

[0151] The standard dimension information corresponding to the target field, or the statistical indicators corresponding to the descriptive information, are filled into the indicator fill position in the query logic structure to form an indicator aggregation suggestion with clear aggregation logic. For example, filling the aggregation function SUM(monthly_sales) into the indicator fill position yields the indicator aggregation suggestion: "To calculate monthly sales, use SUM(monthly_sales) as the aggregation indicator."

[0152] The descriptive information, explicit constraint suggestions, and aggregated indicator suggestions input for data query questions are concatenated in a semantic and logical order to form complete, coherent, and well-constrained query suggestions.

[0153] Following the previous example, the concatenation yields the query suggestion: "Query the monthly sales of stores in City B in year Z." The query table is limited to `dim_store`, the field is limited to store city, and the dimension value can only be B. Monthly sales are calculated using SUM(monthly_sales) as the aggregation metric, generating a directly executable SQL statement.

[0154] By determining the query logic structure through descriptive information and combining it with structured constraints, explicit suggestions are generated. Standard dimension information corresponding to the target field is then populated into the query logic structure to obtain corresponding aggregated suggestion terms. Concatenating the descriptive information, explicit suggestions, and aggregated suggestion terms yields clearly structured and well-defined query suggestions, providing precise guidance for subsequent structured query generation. Furthermore, the use of explicit suggestions and structured constraints avoids misunderstandings caused by mixed information, enabling accurate data query adaptation in zero-sample or low-sample scenarios. This effectively guides the structured query generation process, improving the accuracy and reliability of structured query generation and facilitating adaptation to multi-dimensional data query needs.

[0155] Even after a structured query statement is generated, there may still be grammatical errors, field mismatches, or permission violations. Direct execution of these statements can easily lead to query failures or data security risks. In some optional implementations, after the target structured query statement is generated, a validation and branching mechanism is added to ensure that the executed statement is compliant, usable, and secure, thus guaranteeing a stable and controllable query process.

[0156] Accordingly, the above methods also include: Step d1: Validate the target structured query statement and obtain the statement validation result.

[0157] Step d2: If the verification result indicates that the target structured query statement has passed the verification, then the step of retrieving the target data corresponding to the data query question from the data knowledge base using the target structured query statement is executed.

[0158] Step d3: If the verification result indicates that the target structured query statement has failed the verification, an error message is generated.

[0159] The generated target structured query statement is validated from the dimensions of syntax standardization and logical consistency. Syntax standardization validation includes checking whether keywords, punctuation, and function usage conform to database parsing rules; logical consistency validation includes confirming whether the tables, fields, and dimension values ​​referenced in the target structured query statement actually exist and match constraints.

[0160] After completing the comprehensive validation of the target structured query statement, the corresponding statement validation result is output, which includes either "passed" or "failed". For example, the generated statement `SELECT sales FROM dim_store WHEREcity='B'` passes the validation if it is syntax-compliant, the table and fields exist, and permissions match.

[0161] If the target structured query statement passes the validation, it is submitted to the database engine for execution to extract the target data that matches the conditions from the target data storage unit.

[0162] If the target structured query statement fails validation, the specific reason for the failure can be identified, corresponding error messages can be generated, and output to the front-end interactive interface. For example, if a field name is misspelled, the message "Field 'sales' does not exist in the 'dim_store' table" can be displayed; if a dimension value is out of bounds, the message "Dimension value exceeds the allowed range" can be displayed. Accordingly, the user can adjust the query based on the error messages to regenerate the target structured query statement and retrieve the target data.

[0163] By validating the target structured query statement, invalid and erroneous structured queries can be intercepted before data retrieval, preventing invalid queries from consuming retrieval resources and causing database retrieval anomalies. Simultaneously, precise error alerts can be provided when validation fails, facilitating quick identification of errors so users can make accurate corrections. This allows the system to automatically learn and correct content, achieving self-learning in the generation of structured query statements.

[0164] Step S5033: Based on the query prompts, guide the generation process of the structured query statement to obtain the target structured query statement corresponding to the data query problem.

[0165] The intelligent query system uses the constructed query suggestions to call the large language model to generate structured query statements. Accordingly, within the constraint framework of the query suggestions, the large language model parses the query intent, matches fields and tables, organizes aggregation logic, and outputs the target structured query statement that conforms to the syntax and satisfies the constraints.

[0166] Using the previous example, generate the target structured query statement: SELECT staff_num FROM dim_org WHERE department='R&D Department'.

[0167] Step S504: Using the target structured query statement, retrieve the target data corresponding to the data query question from the data knowledge base. For details, please refer to the relevant descriptions of the corresponding steps in the above embodiments, which will not be repeated here.

[0168] The data retrieval method provided in this embodiment uses structured constraint information to lock the target field, target data storage unit, and dimension field values, accurately guiding the generation direction of query suggestions. It restricts the generation space of structured query statements to the scope of structured constraint information, effectively suppressing problems such as dimension field illusion, table name mismatch, and fictitious dimension field values. This ensures that the target structured query statement is grammatically compliant, logically rigorous, and directly executable, which helps to significantly improve the accuracy of data retrieval results.

[0169] As a specific application embodiment of the present invention, the above data retrieval method is described in conjunction with a specific application scenario. Specifically, it includes: (1) When business personnel enter a natural language question in the front-end interactive interface of the intelligent question system: "Help me see how much money each store in District Y sold each month in year X," the intelligent question system captures the complete text and uses it as the description information for this query.

[0170] (2) The intelligent query system analyzes the description information and identifies the core demand: query the monthly sales of stores in region Y in year X, and determines that the query intention is "to count the monthly sales of stores".

[0171] (3) Retrieve the pre-built data knowledge base, which is constructed based on the data mapping relationship between non-standard dimension information, standard dimension information, and data hierarchical path information. Among them, non-standard dimension information includes: Y area, store, and how much money was sold; standard dimension information includes: Y area, store name, and monthly sales; and data hierarchical path information is group → region → store.

[0172] Based on the query intent, the data knowledge base is retrieved to obtain the target fields that match the query intent: region name, store name, sales amount, and statistical month. The target data storage unit where the target field is located is determined: the business dimension table dim_store_sales.

[0173] (4) Combine the target fields, target data storage units, and description information to generate a compliant target structured query statement (SQL): SELECT Store Name, Month of Statistics, SUM(Sales Amount) AS Monthly Sales Amount FROM dim_store_sales WHERE Region Name = 'Y Region' AND Year of Statistics = X GROUP BY Store Name, Month of Statistics ORDER BY Store Name, Month of Statistics

[0174] (5) Submit the target structured query SQL to the database engine to perform data retrieval, extract data from the dim_store_sales table, and return the structured retrieval results of the target data.

[0175] This embodiment also provides a data retrieval device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0176] This embodiment provides a data retrieval device, such as... Figure 6 As shown, it includes: The acquisition module 601 is used to obtain the description information corresponding to the data query question.

[0177] The retrieval module 602 is used to retrieve, based on the query intent corresponding to the description information, the target field that matches the query intent and the target data storage unit where the target field is located from the pre-built data knowledge base. The data knowledge base is constructed based on data mapping relationships, which are determined based on non-standard dimension information, standard dimension information and data hierarchical path information.

[0178] The query statement generation module 603 is used to generate a target structured query statement corresponding to the data query problem based on the target field, the target data storage unit, and the descriptive information.

[0179] The data retrieval module 604 is used to retrieve the target data corresponding to the data query question from the data knowledge base using the target structured query statement.

[0180] In some alternative implementations, the retrieval module 602 includes: The entity extraction unit is used to extract the query entity corresponding to the description information based on the query intent corresponding to the description information.

[0181] The data knowledge retrieval unit is used to retrieve data knowledge bases based on query entities, obtain the target fields that match the query entities, and the target data storage unit where the target fields are located. The standard dimension information corresponding to the target fields is the query entity.

[0182] In some optional implementations, the data knowledge retrieval unit includes: The information acquisition subunit is used to acquire the various dimension fields of the data knowledge base, the standard and non-standard field values ​​corresponding to each dimension field, and the entity alias corresponding to the queried entity.

[0183] The fuzzy matching subunit is used to perform fuzzy matching between the query entity and the values ​​of each standard field to obtain the query result of the first field.

[0184] The alias matching subunit is used to match entity aliases with non-standard field values ​​to obtain the query results for the second field.

[0185] The hierarchical matching subunit is used to match the query entity with the values ​​of each standard field based on the data hierarchy path information to obtain the query result of the third field.

[0186] The query result fusion subunit is used to determine the target field and the target data storage unit where the target field is located based on the fusion result of the query results of the first field, the second field, and the third field.

[0187] In some optional implementations, the query result fusion subunit is specifically used to: obtain at least one first candidate field and the first candidate data storage unit where each first candidate field is located, based on the fusion result of the first field query result, the second field query result, and the third field query result; and perform disambiguation processing on each first candidate field and each first candidate data storage unit based on a preset disambiguation strategy to obtain the target field and the target data storage unit where the target field is located; wherein, the preset disambiguation strategy includes one or more of the following: text similarity disambiguation strategy, dimension field hierarchy and description information matching degree disambiguation strategy, and historical query field disambiguation strategy.

[0188] In some optional implementations, the query result fusion subunit is specifically used to: obtain at least one second candidate field and the second candidate data storage unit where each second candidate field is located, based on the fusion result of the query result of the first field, the query result of the second field, and the query result of the third field; perform disambiguation processing on each second candidate field and each second candidate data storage unit to obtain at least one third candidate field and the third candidate data storage unit where each third candidate field is located; and determine the target field and the target data storage unit where the target field is located in response to the selection operation on the third candidate field and the third candidate data storage unit.

[0189] In some optional implementations, the query statement generation module 603 includes: The structured processing unit is used to perform structured processing on the target field and the target data storage unit based on a preset constraint template to obtain structured constraint information.

[0190] The prompt word determination unit is used to determine query prompt words based on structured constraint information and descriptive information.

[0191] The generation control unit is used to guide the generation process of structured query statements based on query prompts, and obtain the target structured query statement corresponding to the data query question.

[0192] In some optional implementations, the prompt word determination unit includes: The logical structure determination sub-unit is used to determine the query logical structure of the structured query statement based on the description information, and to set the index padding in the query logical structure.

[0193] The explicit suggestion generation subunit is used to generate explicit suggestions for structured query statements based on structured constraint information.

[0194] The fill sub-unit is used to fill the indicator fill position with the standard dimension information corresponding to the target field to obtain the indicator aggregation prompt words.

[0195] The splicing sub-unit is used to splice descriptive information, explicit prompts, and aggregated index prompts to obtain query prompts.

[0196] In some alternative embodiments, the above-described apparatus further includes: The validation module is used to validate the target structured query statement and obtain the statement validation result.

[0197] The statement generation and execution module is used to execute the steps of retrieving the target data corresponding to the data query question from the data knowledge base using the target structured query statement if the verification result indicates that the target structured query statement has passed the verification.

[0198] The reminder message generation module is used to generate an error reminder message if the validation result indicates that the target structured query statement has failed validation.

[0199] In some alternative implementations, it also includes: The knowledge base building module is used to build a data knowledge base based on data mapping relationships.

[0200] Specifically, the knowledge base construction module includes: The dimension field retrieval unit is used to retrieve multiple dimension fields from various data storage units.

[0201] The field information determination unit is used to determine the standard dimension information, non-standard dimension information, and data hierarchy path information corresponding to each dimension field.

[0202] The mapping building unit is used to construct data mapping relationships based on data hierarchy path information, standard dimension information and non-standard dimension information.

[0203] Index building unit, used to build an inverted index for each dimension field based on data mapping relationships.

[0204] The knowledge base building unit is used to obtain the data knowledge base based on the inverted index of each dimension field.

[0205] In some optional implementations, the knowledge base building module also includes: The mapping update unit is used to update the data mapping relationship of the dimension field in response to configuration operations for non-standard dimension information, and / or in response to changes in the dimension field.

[0206] The data knowledge update unit is used to update the data knowledge base based on the updated data mapping relationship.

[0207] The data retrieval device provided in this embodiment of the invention can execute the data retrieval method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the method. Further functional descriptions of the various modules and units described above are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0208] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0209] The following is a detailed reference. Figure 7 This diagram illustrates a suitable structural schematic for implementing an electronic device according to embodiments of the present invention. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 701, which can perform various appropriate actions and processes based on a program stored in read-only memory (ROM) 702 or a program loaded from memory 708 into random access memory (RAM) 703. The RAM 703 also stores various programs and data required for the operation of the electronic device. The processor 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0210] Typically, the following devices can be connected to I / O interface 705: input devices 706 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 707 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 708 including, for example, magnetic tapes, hard disks, etc.; and communication devices 709. Communication device 709 allows electronic devices to exchange data via wireless or wired communication with other devices. Although Figure 7 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0211] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 709, or installed from a memory 708, or installed from a ROM 702. When the computer program is executed by the processor 701, it performs the functions defined in the data retrieval method of the embodiments of the present invention.

[0212] Figure 7 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of the present invention.

[0213] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the data retrieval method shown in the above embodiments is implemented.

[0214] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0215] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A data retrieval method, characterized by, The method includes: Retrieve the description information corresponding to the data query question; Based on the query intent corresponding to the description information, the target field matching the query intent and the target data storage unit where the target field is located are retrieved from the pre-built data knowledge base. The data knowledge base is constructed based on data mapping relationships, which are determined based on non-standard dimension information, standard dimension information, and data hierarchy path information. Based on the target field, the target data storage unit, and the description information, a target structured query statement corresponding to the data query question is generated. Using the target structured query statement, the target data corresponding to the data query question is retrieved from the data knowledge base.

2. The method of claim 1, wherein, The step of retrieving the target field matching the query intent and the target data storage unit where the target field is located from the pre-built data knowledge base based on the query intent corresponding to the description information includes: Based on the query intent corresponding to the description information, extract the query entity corresponding to the description information; Based on the query entity, the data knowledge base is retrieved to obtain the target field that matches the query entity, and the target data storage unit where the target field is located; The standard dimension information corresponding to the target field is the query entity.

3. The method of claim 2, wherein, The step of retrieving the data knowledge base based on the query entity to obtain the target field matching the query entity and the target data storage unit where the target field is located includes: Obtain the various dimension fields of the data knowledge base, the standard field values ​​and non-standard field values ​​corresponding to each dimension field, and the entity alias corresponding to the queried entity; The query entity is matched with each of the standard field values ​​in a fuzzy manner to obtain the query result for the first field; The entity alias is matched with the non-standard field value to obtain the query result for the second field; Based on the data hierarchy path information, the query entity is matched with each of the standard field values ​​to obtain the query result of the third field; Based on the fusion result of the query results of the first field, the second field, and the third field, the target field and the target data storage unit where the target field is located are determined.

4. The method of claim 3, wherein, The step of determining the target field and the target data storage unit where the target field is located based on the fusion result of the first field query result, the second field query result, and the third field query result includes: Based on the fusion result of the query results of the first field, the query results of the second field, and the query results of the third field, at least one first candidate field and the first candidate data storage unit where each first candidate field is located are obtained; Based on a preset disambiguation strategy, disambiguation processing is performed on each of the first candidate fields and each of the first candidate data storage units to obtain the target field and the target data storage unit where the target field is located. The preset disambiguation strategy includes one or more of the following: text similarity disambiguation strategy, dimension field hierarchy and description information matching degree disambiguation strategy, and historical query field disambiguation strategy.

5. The method of claim 3, wherein, The step of determining the target field and the target data storage unit where the target field is located based on the fusion result of the first field query result, the second field query result, and the third field query result includes: Based on the fusion result of the query results of the first field, the query results of the second field, and the query results of the third field, at least one second candidate field and the second candidate data storage unit where each second candidate field is located are obtained; Disambiguation processing is performed on each of the second candidate fields and each of the second candidate data storage units to obtain at least one third candidate field and the third candidate data storage unit where each of the third candidate fields is located. In response to the selection operation of the third candidate field and the third candidate data storage unit, the target field and the target data storage unit in which the target field is located are determined.

6. The method of claim 1, wherein, The step of generating a target structured query statement corresponding to the data query question based on the target field, the target data storage unit, and the description information includes: Based on a preset constraint template, the target field and the target data storage unit are processed in a structured manner to obtain structured constraint information; Based on the structured constraint information and the description information, query suggestion words are determined; Based on the query suggestion words, the process of generating a structured query statement is guided to obtain the target structured query statement corresponding to the data query question.

7. The method of claim 6, wherein, Based on the structured constraint information and the description information, query suggestion terms are determined, including: Based on the description information, the query logic structure of the structured query statement is determined, and the index filling position is set in the query logic structure; Based on the structured constraint information, explicit prompts for the structured query statement are generated; Fill the standard dimension information corresponding to the target field into the indicator filling position to obtain the indicator aggregation prompt words; The query suggestion is obtained by concatenating the description information, the explicit suggestion words, and the aggregated suggestion words.

8. The method of claim 6, wherein, Also includes: The target structured query statement is validated to obtain the statement validation result; If the verification result indicates that the target structured query statement passes the verification, then the step of retrieving the target data corresponding to the data query question from the data knowledge base using the target structured query statement is executed; If the verification result indicates that the target structured query statement has failed verification, an error message will be generated.

9. The method according to any one of claims 1 to 8, characterized in that, The data knowledge base is constructed based on data mapping relationships, including: Retrieve multiple dimension fields from each data storage unit; Determine the standard dimension information, non-standard dimension information, and data hierarchy path information corresponding to each of the aforementioned dimension fields; Based on the data hierarchy path information, the standard dimension information, and the non-standard dimension information, the data mapping relationship is constructed; Based on the data mapping relationship, construct an inverted index for each of the dimension fields; The data knowledge base is obtained based on the inverted index of each of the aforementioned dimension fields.

10. The method of claim 9, wherein, Also includes: In response to a configuration operation for the non-standard dimension information, and / or, in response to a change in the dimension field, update the data mapping relationship of the dimension field; The data knowledge base is updated based on the updated data mapping relationship.

11. A data retrieval device, characterized in that, The device includes: The acquisition module is used to obtain descriptive information corresponding to the data query question; The retrieval module is used to retrieve, based on the query intent corresponding to the description information, the target field that matches the query intent and the target data storage unit where the target field is located from a pre-built data knowledge base. The data knowledge base is constructed based on data mapping relationships, which are determined based on non-standard dimension information, standard dimension information and data hierarchy path information. The query statement generation module is used to generate a target structured query statement corresponding to the data query question based on the target field, the target data storage unit, and the description information. The data retrieval module is used to retrieve the target data corresponding to the data query question from the data knowledge base using the target structured query statement.

12. An electronic device, characterized in that, include: A memory and a processor are communicatively connected, the memory stores computer instructions, and the processor executes the computer instructions to perform the data retrieval method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the data retrieval method according to any one of claims 1 to 10.

14. A computer program product, characterized in that, Includes computer instructions for causing a computer to perform the data retrieval method according to any one of claims 1 to 10.

Images