Business data processing method and device, electronic equipment and storage medium
By constructing a query processing method based on a finely tuned proprietary large model and business knowledge base, the problem of low query accuracy caused by user input bias was solved, and the automatic conversion from natural language to structured query was realized, improving the accuracy and automation level of query results.
Patent Information
- Application Number
- CN202511189475.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-22
- Publication Date
- 2025-12-09
AI Technical Summary
In existing technologies, the natural language text input by users may be unclear, vague, or ambiguous in the generated query statements, resulting in query results that deviate from expectations and have low accuracy.
By acquiring a finely tuned proprietary large model, business knowledge base, and business database, a query semantic vector is generated, table data query operations are performed, initial prompt words are generated, and a structured query statement to be executed is generated by combining the data query question and the business database. The query operation is then executed by a preset executor, and finally, the results are presented.
It achieves efficient conversion from natural language questions to structured query statements, improves the accuracy and automation of query results, ensures the matching degree between query results and user intent, and enhances the user interaction experience.
Smart Images

Figure CN121092541A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing technology, and in particular to a business data processing method, processing device, electronic device, and storage medium. Background Technology
[0002] As enterprises continue their digital transformation, business data is expanding in terms of source, type, and scale. Large amounts of structured data, such as transaction records, customer information, financial data, and system logs, are uniformly stored in the enterprise's internal business database. To support business analysis, performance monitoring, and management, users typically raise specific data query questions based on their business needs and perform retrieval operations in the database through query interfaces or structured query languages (such as SQL) to obtain the required detailed data, statistical indicators, or summary results.
[0003] In existing technologies, the implementation paths for business data queries mainly fall into two categories. One is an automatic conversion method from natural language to structured query statements based on a general language model: the user's natural language query question is concatenated with preset prompts and input into the general language model. The model then generates a structured query statement to be executed, which is subsequently executed by the database engine and the query results are returned. The other is a rule-driven query generation method, which typically relies on keyword extraction, statement templates, or mapping tables. This method matches the user's question with pre-defined query fragments or complete statements, and completes the final query statement construction and execution through parameter filling.
[0004] However, existing technologies rely on natural language text input from users. When user input is unclear, ambiguous, or uses ambiguous terminology, the generated query may fail to accurately reflect the user's query intent, causing the returned query results to deviate from expectations. Therefore, there is an urgent need for a business data processing method to address the technical problem of low accuracy in business data query results in existing technologies. Summary of the Invention
[0005] This application provides a business data processing method, processing device, electronic device, and storage medium to improve the matching degree between query statements and user intent, thereby enhancing the accuracy of query results.
[0006] In a first aspect, embodiments of this application provide a business data processing method, including:
[0007] Obtain the finely tuned proprietary large model, business knowledge base, and business database;
[0008] Obtain the data query question and generate a query semantic vector based on the data query question;
[0009] Based on the query semantic vector, perform a table data query operation in the business knowledge base to obtain data table information;
[0010] Based on the information in the data table, generate initial prompt words;
[0011] Based on the initial prompt words, the data query question, the business database, and the finely tuned proprietary large model, a structured query statement to be executed is obtained;
[0012] The structured query statement to be executed is input into a preset executor. When the preset executor executes the structured query statement to be executed successfully, a query operation is performed from the business database to obtain preliminary result data.
[0013] The preliminary results data are processed to obtain the data to be presented.
[0014] In one possible implementation, obtaining the finely tuned proprietary large model includes:
[0015] Obtain the open-source basic large model and the public dataset of structured query statements, and perform initial fine-tuning on the open-source basic large model based on the public dataset of structured query statements to obtain the initial fine-tuned proprietary large model;
[0016] The initial fine-tuned proprietary large model is then fine-tuned a second time based on a pre-built self-built business dataset to obtain the fine-tuned proprietary large model.
[0017] In one possible implementation, acquiring the business knowledge base includes:
[0018] Obtain business data;
[0019] The data table name, data table fields, data types, field types, field remarks, and field lengths are determined based on the business data.
[0020] The data table name, data table fields, data types, field types, field remarks, and field lengths are used to generate a knowledge base table document as a business knowledge base according to a preset database table structure.
[0021] In one possible implementation, the step of performing a table data query operation in the business knowledge base based on the query semantic vector to obtain data table information includes:
[0022] Based on the query semantic vector, generate search suggestion terms;
[0023] Based on the search suggestions and the preset general model, table data query operations are performed in the business knowledge base to obtain a paradigm output result containing the target table structure information from the business knowledge base;
[0024] The output results of the above paradigm are used to perform data selection operations using a table name similarity hard matching algorithm to obtain data table information.
[0025] In one possible implementation, the presentation processing based on the preliminary result data to obtain the data to be presented includes:
[0026] Based on the preliminary results data, a mapping and parsing process is performed to obtain key-value pair structure data;
[0027] The structure data is parsed and concatenated based on the key values to obtain plain text markup language structure data.
[0028] In one possible implementation, after inputting the structured query statement to be executed into the preset executor, the method further includes:
[0029] When the execution result of the preset executor of the structured query statement to be executed fails, the structured query statement to be executed is re-inputted into the fine-tuned proprietary large model for adjustment processing to obtain the adjusted structured query statement to be executed. The adjusted structured query statement to be executed is then updated as the structured query statement to be executed, and the process jumps to the step of inputting the structured query statement to be executed into the preset executor.
[0030] In one possible implementation, after performing presentation processing based on the preliminary result data to obtain the data to be presented, the method further includes:
[0031] The query semantic vector is stored in a preset vector database as a historical query semantic vector, and the data to be presented is stored in a preset cache area as historical cache data.
[0032] Obtain the new query question and generate a new query semantic vector based on the new query question;
[0033] The new query semantic vector is used for matching in the preset vector database to obtain the matching result.
[0034] When the matching result is detected to meet the preset matching success conditions, the historical cached data corresponding to the historical query semantic vector that was successfully matched is determined as the data to be presented now.
[0035] Secondly, embodiments of this application provide a business data processing apparatus, including:
[0036] The acquisition module is used to acquire the finely tuned proprietary large model, business knowledge base, and business database.
[0037] The semantic understanding module is used to obtain the data query question and generate a query semantic vector based on the data query question;
[0038] The semantic understanding module is used to perform table data query operations in the business knowledge base according to the query semantic vector to obtain data table information.
[0039] The semantic understanding module is also used to generate initial prompt words based on the data table information;
[0040] The query statement generation module is used to generate a structured query statement to be executed based on the initial prompt words, the data query question, the business database, and the fine-tuned proprietary big model.
[0041] The query module is used to input the structured query statement to be executed into a preset executor. When the preset executor executes the structured query statement to be executed successfully, a query operation is performed from the business database to obtain preliminary result data.
[0042] The data presentation module is used to process the preliminary result data to obtain the data to be presented.
[0043] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor;
[0044] The memory stores computer-executed instructions;
[0045] The processor executes computer execution instructions stored in the memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.
[0046] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.
[0047] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.
[0048] This application provides a business data processing method, processing device, electronic device, and storage medium. By acquiring a finely tuned proprietary large model, a business knowledge base, and a business database, it constructs a semantic understanding and data retrieval environment oriented towards business scenarios, ensuring that subsequent query processing has business context adaptability and data source integrity. By acquiring the data query question and generating a query semantic vector, it achieves structured semantic encoding of the user's natural language intent, improving the understandability of query requirements. Furthermore, based on the query semantic vector, it performs table data query operations in the business knowledge base to obtain data table information, enabling the model to accurately identify the range of data tables involved in the user's query question, improving the context relevance of downstream query generation. The operation of generating initial prompt words based on data table information helps to provide the table structure prior information required by the model when generating structured query statements, thereby enhancing the accuracy and relevance of the generated results. Combining the initial prompt words, the original query question, the business database, and the proprietary large model to generate a structured query statement to be executed, it realizes the mapping from semantics to structured language under complex query targets. The generated query statement is input to a preset executor for execution, and the subsequent database operations are driven based on the success or failure of the execution, ensuring the stability of the execution path and the effectiveness of the result acquisition. Finally, the preliminary results data are processed to generate structured and readable results, achieving a complete closed loop from user intent to data feedback. Overall, this application achieves automatic conversion from natural language questions to structured data queries, effectively solving the problem of low query accuracy caused by user input bias in existing technologies, achieving a level of automation in business data retrieval, and enhancing the accuracy of query results and user interaction experience. Attached Figure Description
[0049] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0050] Figure 1 A flowchart illustrating the business data processing method provided in this application;
[0051] Figure 2 This is a flowchart illustrating a method for adjusting a structured query statement to be executed, as provided in an embodiment of this application.
[0052] Figure 3 This is a schematic flowchart of a method for quickly matching and obtaining data to be presented, provided in an embodiment of this application.
[0053] Figure 4 A schematic diagram of the business data processing device provided in this application;
[0054] Figure 5 A schematic diagram of the structure of the electronic device provided in this application.
[0055] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation
[0056] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.
[0057] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.
[0058] In existing technologies, queries are typically generated from user-inputted natural language to retrieve relevant information. However, user input often suffers from issues such as non-standard expression, non-standard terminology, or ambiguous semantics, leading to queries that do not match actual needs. This, in turn, affects query efficiency and the accuracy of data retrieval.
[0059] Based on the aforementioned technical problems and needs, this application proposes a business data processing method aimed at improving the accuracy and automation of natural language query requests executed in a business database. Specifically, it acquires a finely tuned proprietary large-scale model, a business knowledge base, and a business database to construct a basic query support environment. When a user inputs a data query question, a corresponding query semantic vector is generated based on the query question, thereby transforming the natural language query intent into a vector expression recognizable by the model. Subsequently, based on this query semantic vector, table data queries are performed in the business knowledge base to identify and locate data table information related to the query question. Based on the acquired data table information, initial prompt words are further generated to form the model input context together with the original query question.
[0060] Subsequently, using the initial prompt, the data query question, the business database, and the finely tuned proprietary large model, a structured query statement to be executed is generated that can be recognized by the database execution engine. The generated query statement is input into a preset executor module. If the executor successfully runs the statement, it performs the query operation from the business database and retrieves preliminary result data. Finally, the preliminary result data is processed to obtain formatted, visualized, or structured data for easier user understanding and use. This application achieves efficient conversion from natural language questions to structured query statements and accurately acquires and displays business data.
[0061] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.
[0062] Figure 1 A flowchart illustrating the business data processing method provided in this application is shown below. Figure 1 As shown, the method includes:
[0063] S11, retrieve the finely tuned proprietary large model, business knowledge base, and business database.
[0064] In this embodiment, to ensure the consistency and accuracy of subsequent data query parsing, mapping, and execution, multiple components are first loaded and initialized. These include acquiring a fine-tuned proprietary large-scale model, a business knowledge base, and a business database. The fine-tuned proprietary large-scale model refers to a model trained and adjusted on a specific task or dataset based on a general language model. It possesses contextual understanding and semantic mapping capabilities, enabling it to understand business data query questions and generate semantic expression vectors, ensuring the professionalism and adaptability of subsequent query semantic processing and structured query statement generation. The business knowledge base stores metadata information related to the business database structure, such as table names, field names, field types, and field comments, forming an intermediate knowledge layer to support semantic-to-structure mapping. The business knowledge base can be seen as an abstract and standardized expression of the business database structure, facilitating the subsequent retrieval of target data table information through semantic parsing and retrieval operations, thereby improving the accuracy of structure mapping. The business database stores actual business data, covering transaction records, customer information, log tracking, etc., and serves as the underlying data source for data querying and result return. During the subsequent execution of structured query statements, the business database provides direct data reading and processing support. This embodiment constructs a basic business data processing environment covering semantic understanding, structure awareness, and data access by acquiring three components, achieving flexible support for multiple scenarios and types of business data.
[0065] S12, obtain the data query question, and generate a query semantic vector based on the data query question.
[0066] In this embodiment, a data query question is obtained, and a query semantic vector is generated based on the data query question. A data query question refers to a query request submitted by a user in natural language, typically reflecting an intent to statistically analyze, filter, correlate, or aggregate business data, and possessing an open semantic structure and high expressive diversity. For example, a user might ask questions like "Compare sales figures for each region over the past three months" or "Query the top ten products with the most customer complaints this quarter." Upon receiving this natural language input, it needs to be prepared as a structured representation of the query instruction to be processed. To achieve semantic understanding and vector encoding, the data query question is vectorized. For example, an embedding model is used to map the query question to a fixed-dimensional high-dimensional semantic vector (e.g., a 1024-dimensional floating-point array) to capture key entities, query intent, and contextual relationships. This embodiment, by transforming unstructured text input into a standardized semantic representation, achieves a seamless connection between natural language and structured processing, enhancing the consistency and operability of subsequent stages in matching, generation, and execution. Furthermore, to improve the consistency and robustness of semantic vectors, preprocessing operations can be performed on the data query problem before generating query semantic vectors, such as unified character encoding, stop word filtering, time expression standardization, and named entity recognition, to reduce interference caused by semantic drift.
[0067] S13, perform table data query operations in the business knowledge base based on the query semantic vector to obtain data table information.
[0068] In this embodiment, table data queries are performed in the business knowledge base based on the query semantic vector to obtain the data table information required for subsequent generation of structured query statements. The purpose of this embodiment is to establish a structural linkage between natural language input and structured data, that is, to guide table structure selection through semantic understanding, enabling subsequent data access processes to have a structural basis and business interpretability. The table data query operation here refers to the structure retrieval process oriented towards the metadata layer. The business knowledge base, as a carrier of structured knowledge, contains the structural definition information of multiple business tables, including table names, field names, field types, field descriptions, and their semantic roles in the business system. By performing structure-level queries in the business knowledge base, the path from the user's query semantics to the target data structure is located, thereby constructing a complete structural context for the data query process.
[0069] The purpose of this operation is to guide natural language queries to the relevant business structure through a preprocessed semantic indexing mechanism, reducing structural mismatches caused by ambiguous query content or inconsistent terminology. In summary, this embodiment achieves the transition from language expression to structural definition by mapping and retrieving query semantic vectors in the business knowledge base to obtain data table information associated with the query target. This provides a data structure foundation for the subsequent generation of structured query statements, ensuring that the query task possesses both semantic integrity and structural executability, thus improving the accuracy of structured query statement generation.
[0070] S14. Generate initial prompt words based on the data table information.
[0071] In this embodiment, the initial prompt words refer to the input content used to guide the language model in generating structured query statements. For example, they may include the natural language query question itself and the corresponding table structure information. Because natural language expression is inherently uncertain, while database structures are highly constrained, relying solely on semantic input often leads to query statements generated by the model containing field errors, structural omissions, or logical ambiguities. By explicitly introducing table information into the prompt words, the language model can be provided with a complete field context and structural scope, thereby improving the model's ability to correctly infer field selection, logical operations, and query objectives. This embodiment, by generating initial prompt words containing structural information and semantic intent, bridges the format from natural language questions to structured generation input, achieving the integration of semantic content and data structure. This ensures that the subsequent structured query statement generation process has a solid input foundation in terms of field constraints and semantic consistency.
[0072] For example, initial suggestion words can be obtained through an open-source real-time analytical database, such as Doris Database Assistant. Doris Database Assistant can provide functions such as parsing and displaying table structures, suggesting field selections, or providing semantic completion hints, thereby guiding users to accurately set suggestion words that match the current query target. Based on the table structure information obtained during the interaction, the suggestion words clearly indicate the expected field names, data aggregation logic, filtering conditions, or expected output format, forming query guidance information with contextual constraints.
[0073] S15: Based on the initial prompts, data query questions, business database, and the finely tuned proprietary large model, the structured query statement to be executed is obtained.
[0074] In this embodiment, a structured query statement to be executed is generated based on the initial prompts, the data query question, the business database, and a finely tuned proprietary large-scale model. The purpose is to transform the composite input, which integrates semantic expression and structural information, into an executable structured query statement, typically represented in standardized SQL (Structured Query Language) syntax, for direct execution in the business database to obtain data results. The structured query statement to be executed refers to the query instruction that conforms to the database syntax requirements and is output after processing by the language model. Its content includes elements such as the target table, query fields, filtering conditions, grouping methods, and sorting rules, and is consistent with the data structure in the business database. The generation of this statement is accomplished through a finely tuned proprietary large-scale model. This model possesses natural language parsing and structured statement construction capabilities, and after targeted training for business data scenarios, it can understand the relationship between query intent and database structure, and organize query logic accordingly.
[0075] In the implementation, the initial prompts serve as model input, providing structural constraints and problem context, while the original data query question provides the user's original semantic intent. These two types of information jointly participate in context modeling within the language model. Through encoding and decoding mechanisms, the model generates query statements that best meet the problem requirements and do not exceed structural boundaries. Although the business database is not directly part of the input data stream in this process, its table structure information forms the basis of constraints on the generated results. The purpose of this step is to achieve accurate conversion between natural language and structured query language, reduce the high difficulty of traditionally manually writing query statements, and ensure that the generated statements possess structural accuracy, semantic integrity, and execution feasibility. By integrating the table structure information from the initial prompts with the query target semantics from the data query question, and leveraging the finely tuned proprietary large-scale model's learning capabilities in both language and structure dimensions, highly usable structured query statements can be output. This embodiment completes the transformation process from natural language to structured query language by using initial prompts, data query questions, business database structure information, and a finely tuned proprietary large model. It realizes semantic generation driven by structure, effectively improving the accuracy of statement generation, execution success rate, and scenario adaptability, and providing a clear and executable instruction foundation for subsequent data extraction and result presentation.
[0076] S16, input the structured query statement to be executed into the preset executor. When the preset executor executes the structured query statement to be executed successfully, execute the query operation from the business database to obtain preliminary result data.
[0077] In this embodiment, after generating the structured query statement to be executed, the executability and validity of the statement are verified, and the data access operation is actually triggered. To this end, the structured query statement to be executed is input into a preset executor. When the preset executor successfully executes the structured query statement, the query operation is performed from the business database to obtain preliminary result data. The preset executor is a component used to receive and execute structured query statements, typically integrating interface capabilities for communication with the business database, syntax compliance verification mechanisms, and result packaging modules. In implementation, this executor can be implemented based on a standard database access framework to ensure the stability and security of the instruction's execution. After receiving the structured query statement, the executor first performs a parsing check on the statement, including basic verifications such as whether table fields exist and whether parameters are complete; if the verification passes, the statement is sent to the business database for formal execution.
[0078] The business database, serving as the storage carrier of structured data, receives query requests from the executor. Based on the target table, filtering conditions, aggregation logic, and other operational parameters contained in the query statement, it executes the corresponding data retrieval operation and returns the query results. These results can be a collection of data records from a single table or multiple tables, typically expressed in a structured table format, containing the correspondence between field names and data values. This query result is defined as preliminary result data, serving as the input source for subsequent presentation processing. This embodiment ensures that the generated structured query statement not only has logical rationality in the language model but also passes execution validation in the database environment and produces valid data output. By using the executor as an intermediary control point between the statement and the database, it can capture anomalies such as syntax errors, insufficient permissions, and missing fields during execution, providing a trigger basis for possible statement repair mechanisms. In the case of successful execution, it ensures that the data retrieval behavior has been completed within the structural boundaries, obtaining result data that matches the semantic intent of the original natural language question. This establishes a clear and reliable result foundation for subsequent data parsing and display operations.
[0079] S17. Based on the preliminary results data, perform presentation processing to obtain the data to be presented.
[0080] In this embodiment, the preliminary result data retrieved from the business database is further processed to ensure that the information ultimately presented to the user has good readability and business interpretability. Specifically, the structure and content of the preliminary result data are first parsed and reorganized, typically including the identification and classification of information such as field names, data values, and data types, to establish a clear data presentation structure. Based on this, the data content can be formatted using preset display templates or visualization rules, such as converting the results into tables, charts, or plain text to adapt to the display needs of different terminals or the reading preferences of users. This embodiment, through semantic restoration and structural reorganization of the preliminary result data, achieves the conversion from the original database return format to a user-friendly display format, improving the understandability and business adaptability of the query results. Furthermore, the processed data to be presented is easier to cache, compare, or reuse subsequently, and its practicality and interactivity are improved.
[0081] In one embodiment, the process of obtaining the finely tuned proprietary large model in step S11 above will be further explained here. Based on the above embodiment, it includes:
[0082] S1111: Obtain the open-source basic large model and the public dataset of structured query statements, and perform initial fine-tuning of the open-source basic large model based on the public dataset of structured query statements to obtain the initial fine-tuned proprietary large model.
[0083] S1112, perform secondary fine-tuning on the initial fine-tuned proprietary large model based on the preset self-built business dataset to obtain the fine-tuned proprietary large model.
[0084] In this embodiment, to enable the language model to generate structured query statements and adapt to the semantic features of specific business scenarios, the open-source basic model needs to be fine-tuned in stages. First, the open-source basic model and a public dataset of structured query statements are acquired. Based on this public dataset, the open-source basic model is initially fine-tuned. Then, a second fine-tuning is performed using a pre-built business dataset, ultimately resulting in a well-adapted, stable, and finely tuned proprietary model. The open-source basic model refers to a pre-trained language model with a large number of parameters and general language modeling capabilities, such as the QwenCoder-14B model. It has been pre-trained on a large-scale natural language corpus and possesses contextual understanding and text generation capabilities, but it has not been optimized for structured query statements. Therefore, to enable this type of model to analyze and generate structured query statements from natural language questions, a public dataset of structured query statements needs to be set up for initial fine-tuning. This type of public dataset typically contains a large number of paired samples, each consisting of a natural language question and a corresponding SQL query. The structure is rich, covering various SQL patterns such as multi-table joins, aggregate functions, filtering logic, and sorting statements.
[0085] The initial fine-tuning employs supervised learning, typically inputting text pairs into the model for training and calculating cross-entropy loss using the target query. Parameters are adjusted across multiple training epochs. Lightweight fine-tuning frameworks such as LoRA (Low-Rank Adaptation) can be used to reduce resource consumption and improve deployment efficiency. After the initial fine-tuning, a proprietary large model is obtained. To further enhance the model's adaptability to specific business domain terminology, query patterns, and field semantics, a secondary fine-tuning is performed based on a self-built business dataset. This dataset consists of common natural language queries and corresponding structured query statements from the target enterprise's actual business, covering specific table structures, field naming rules, query preferences, and logical expressions. The dataset can be constructed from historical query logs, manually labeled samples, or semantic-structure pairs automatically generated from a business knowledge base. The secondary fine-tuning continues training based on the initial proprietary large model, maintaining basic SQL capabilities while aligning the model's language space with real-world scenarios. In summary, this embodiment establishes a transition between general capabilities and specific adaptations through a two-stage fine-tuning mechanism, thereby enhancing structured query capabilities. The initial fine-tuning provides basic syntax construction and logical analysis capabilities, while the secondary fine-tuning injects specific semantic preferences and structural boundaries, ultimately forming a finely tuned proprietary large model with high accuracy, high context matching, and business relevance, providing a stable model foundation for natural language-driven structured data access.
[0086] In one embodiment, the acquisition of the business knowledge base in step S11 above will be further explained. Based on the above embodiment, it includes:
[0087] S1121, Obtain business data;
[0088] S1122, Determine the data table name, data table fields, data type, field type, field remarks, and field length based on business data;
[0089] S1123 generates a knowledge base table document based on the data table name, data table fields, data types, field types, field remarks, and field lengths according to the preset database table structure, serving as the business knowledge base.
[0090] In this embodiment, to ensure that subsequent semantic retrieval and structured query tasks can be carried out based on accurate and complete database structure information, the current business data needs to be structured and parsed, and a knowledge base table document needs to be constructed in a unified format as the basis for matching the semantic vector of the subsequent query with the table structure information. Specifically, the business data is first obtained. Business data usually includes database metadata files, data dictionaries, historical interface documents, database exported structure scripts, etc., and may also come from the metadata interface provided by the database management system (such as MySQL). By connecting to the database and reading its metadata table, all data tables defined in the current database and their structure information can be automatically extracted, reducing the error of manual input. Subsequently, the business data is parsed, automatically extracting information from multiple dimensions related to the table structure. These include: table name (the logical table name, used to identify the subject or purpose of a data set), table fields (the names of each column, used to describe specific business attributes), data types (such as integer, floating-point, text, date, etc., used to define the data representation of fields), field types (such as primary keys, foreign keys, index fields, etc., used to assist in database logical constraints), field comments (used to explain the meaning of fields and improve semantic clarity), and field lengths (used to define storage capacity). These dimensions collectively constitute the semantic boundaries of the table structure.
[0091] After extracting the above information, it is organized into a standardized knowledge base table document according to a preset database table structure template. This table document uses a structured format (such as JSON or Markdown tables) to uniformly represent the structural information of each data table, facilitating table-level or field-level semantic matching in general or proprietary large models. The knowledge base table document not only contains the technical attributes of the fields but also integrates the business semantic information carried by the field notes, thereby providing higher accuracy and relevance during semantic retrieval. This embodiment, by systematically extracting database structure information and constructing a standardized business knowledge base, achieves a bridge between semantic intent and the underlying table structure in natural language queries. It reduces the information loss problem in the semantic-to-structure mapping process; provides semantically interpretable and structurally clear knowledge support for the model; improves the efficiency and accuracy of query table information retrieval; and enhances the reliability of the entire business data processing flow.
[0092] For example, the dynamic weekly initialization table in the business database is used as the base data table. This table contains multiple fields related to billing dates and periodic information to support subsequent time-dimensional analysis and statistical modeling. The main fields in this table include bill_year (billing year), bill_month (billing month), bill_day (billing day), week_index (dynamic annual sequence number), week_sequence (dynamic weekly sequence number), and quarter (natural quarter). During the table structure construction process, the table name, field names, field types, nullability, field length, and field comment information are extracted according to preset field attributes and organized into a knowledge base table document according to a preset standard format, serving as part of the business knowledge base. This document will subsequently be used to drive the configuration of prompt words for generating structured query statements from the proprietary large model and the table field matching process. The specific structure is shown in the table below:
[0093] Table 1 Dynamic Weekly Initialization Table
[0094]
[0095] In one embodiment, the acquisition of data table information in step S13 is implemented in a manner described herein. Based on the above embodiment, it includes:
[0096] S131, Generate search suggestion words based on the query semantic vector;
[0097] S132, perform table data query operations in the business knowledge base based on the search prompts and the preset general model, so as to obtain the paradigm output results containing the target table structure information from the business knowledge base;
[0098] S133 uses a table name similarity hard matching algorithm to select data from the output of the normal form and obtain data table information.
[0099] In this embodiment, search suggestions are generated based on the query semantic vector. The query semantic vector is a vector representation formed by semantically encoding the natural language query input by the user, containing multi-dimensional semantic information such as entities, attributes, and actions implied in the query statement. Based on this vector, semantically representative keywords or phrases are extracted using semantic deconstruction and vector inverse mapping techniques to form search suggestions suitable for search engine or large-scale model retrieval tasks. These suggestions not only cover explicit information in the user's original language expression but also reveal the implicit intent behind it, such as field meaning or business context.
[0100] After generating search suggestions, a pre-defined general model is invoked to perform table data queries on the business knowledge base based on these suggestions. This general model serves as a semantic understanding and text reasoning engine, analyzing and outputting candidate results that may contain target table structure information through semantic matching between the suggestions and the content of the business knowledge base. Since the business knowledge base typically exists in the form of structured tabular documents, the general model can perform contextual matching between field names, field notes, and suggestions in the tables, and output normalized paradigmatic structure information, such as structured candidate results composed of matching table names, main fields, and related fields.
[0101] After obtaining the candidate paradigm output, a hard matching algorithm for table name similarity is further used to perform data selection operations, determining the final table information from multiple candidate results. The hard matching algorithm for table name similarity is a rule or model combination method based on character edit distance, root word extraction, abbreviation mapping, or word embedding cosine similarity, used to calculate the degree of matching between the table name in the user's intent and the actual table name in the business knowledge base. Hard matching emphasizes the uniqueness and certainty of the result, eliminating ambiguity that may be introduced by fuzzy matching, thereby ensuring high accuracy of the table information relied upon by subsequent structured queries. This embodiment constructs a multi-level semantic retrieval chain from query semantic vector to prompt words, prompt words to general large model retrieval, general large model paradigm output to hard matching confirmation, realizing the mapping between user natural language intent and specific database structure. This improves the accuracy of table structure retrieval under complex query intents and enhances the practicality and generalization ability of the large model in the process of understanding structured data.
[0102] In one embodiment, the data to be presented obtained in step S17 above is provided as an implementation method. Based on the above embodiment, it includes:
[0103] S171, Based on the preliminary result data, perform mapping parsing to obtain key-value pair structure data;
[0104] S172, based on the key values, the structure data is parsed and concatenated to obtain plain text markup language structure data.
[0105] In this embodiment, the preliminary result data is mapped and parsed to obtain key-value pair structured data. The preliminary result data is the query result returned after a structured query is executed in the business database. It is typically represented as a tabular data set containing several fields and their corresponding field values. At this stage, field identification and semantic mapping operations are performed on this tabular structured data, mapping the field name and its corresponding field value in each data record to a set of key-value pairs. For example, for result data containing the fields "customer name" and "transaction amount," it is parsed into the form {customer name: Zhang San, transaction amount: 10000 yuan}. This key-value pair structure not only preserves the field semantic information of the original data but also has high scalability, facilitating subsequent data transformation and display logic calls.
[0106] After obtaining the key-value pair structured data, the data is parsed and concatenated based on preset formatting rules to generate plain text markup language structured data. Here, plain text markup language structured data refers to text data in a formatted way, such as JSON, HTML, or Markdown, expressing key-value pairs. In this process, the appropriate text format type is selected according to the requirements of the target display platform or interface, and field sorting, data indentation, and field naming styles (such as camelCase or underscore) are standardized to generate a result document that conforms to display standards. This embodiment achieves a unified data transmission format between business systems and front-end interfaces or external service systems by converting structured data results into key-value pairs and then formatting them into plain text markup language, improving overall integration capabilities and cross-platform compatibility; it also simplifies the processing burden on the front end during parsing and display, and improves the user's intuitive understanding of the data results.
[0107] Figure 2 This is a flowchart illustrating a method for adjusting a structured query statement to be executed, provided in an embodiment of this application. Based on the above embodiment, after inputting the structured query statement to be executed into a preset executor, the method includes:
[0108] S21, when the execution result of the pre-set executor is a failed execution of the structured query statement to be executed, the structured query statement to be executed is re-entered into the fine-tuned proprietary large model for adjustment processing to obtain the adjusted structured query statement to be executed.
[0109] S22, update the adjusted structured query statement to be executed to the structured query statement to be executed, and jump to the step of inputting the structured query statement to be executed into the preset executor.
[0110] In this embodiment, when the execution result of the structured query statement to be executed by the preset executor fails, an error repair mechanism is automatically triggered to ensure the continuity and robustness of the query process. By reusing the fine-tuned proprietary large model to adjust the original structured query statement, adaptive correction and retry execution of erroneous statements are achieved, thereby improving the fault tolerance and availability of the overall business data processing. Specifically, the preset executor is a logical component used to execute structured query statements in the business database. If an execution failure occurs during the execution of the structured query statement to be executed, such as syntax errors, field mismatches, table non-existence, or logical condition conflicts, it will be determined that the structured query statement cannot complete the expected data retrieval task. At this time, instead of directly terminating the process, the failed structured query statement is re-entered into the fine-tuned proprietary large model as input. The fine-tuned proprietary large model has been fully trained on the structured query statement related dataset and has the ability to identify and repair common erroneous queries. The model will adjust the previous structured query statement to be executed based on the semantic structure of the current query statement, known error features, and historical semantic vector information to obtain the adjusted structured query statement to be executed.
[0111] Subsequently, the adjusted structured query statement to be executed is updated to a new structured query statement to be executed, ensuring that the subsequent executor obtains the corrected query version. After the update is completed, it automatically jumps back to the execution phase of the structured query statement, that is, the preset executor is called again to execute the new structured query statement. This setting based on the failure retry mechanism enables self-recovery when facing abnormal structured statements, reduces user waiting or manual intervention caused by query failures, and improves the automation level and execution success rate of the query process. This embodiment achieves automatic backtracking and model correction operations in the case of execution failure by repairing the structured statement while preserving semantics, improving the stability of business data processing and the integrity of query tasks. At the same time, it provides a feedback basis for subsequent construction of query statement optimization and reinforcement learning, further enhancing the model's adaptability in complex query environments.
[0112] Figure 3 A schematic flowchart illustrating a method for quickly matching and obtaining data to be presented, provided in an embodiment of this application. Based on the above embodiment, it includes:
[0113] S31, store the query semantic vector as a historical query semantic vector in the preset vector database, and store the data to be presented as historical cache data in the preset cache area;
[0114] S32, Obtain the new query question and generate a new query semantic vector based on the new query question;
[0115] S33, perform matching processing in the preset vector database based on the new query semantic vector to obtain the matching result;
[0116] S34. When the matching result is detected to meet the preset matching success condition, the historical cache data corresponding to the historical query semantic vector that was successfully matched is determined as the data to be presented now.
[0117] In this embodiment, to improve the response speed and resource utilization efficiency of business data processing in repetitive query scenarios, a vector matching and caching mechanism is implemented to achieve rapid response to similar query problems. This mechanism determines the historical cache hit status based on semantic vector matching, thereby directly outputting cached query results without regenerating structured query statements or executing database queries. First, after the current query task is completed, the currently generated query semantic vector is stored in a preset vector database. A vector database is a storage structure that supports high-dimensional semantic vector retrieval and can efficiently perform similarity calculations and similar semantic vector matching operations. Simultaneously, the data to be presented is synchronously stored in a preset cache area as historical cache data. The cache area can adopt a key-value mapping structure, where the key is a unique identifier or compressed representation of the query semantic vector, and the value is the corresponding query result data, ensuring fast access in subsequent queries.
[0118] When a new data query is received, the same processing flow as before is first executed, namely, generating a new query semantic vector using a finely tuned proprietary large model or related modules. Subsequently, this new query semantic vector is used for matching in a preset vector database. The matching process can employ a matching algorithm based on vector space distance (such as cosine similarity, Euclidean distance, etc.) to sort historical vector data by similarity. If a historical query semantic vector in the matching results has a similarity exceeding a preset successful matching condition (such as reaching a preset threshold), it is considered a cache hit. In this case, the historical cached data corresponding to the successfully matched historical query semantic vector is directly determined as the data to be presented for the current query. This embodiment, by combining semantic vector representation with vector database retrieval, achieves approximate matching and result reuse at the semantic level for query questions, improving query efficiency and reducing resource waste caused by redundant computation.
[0119] Figure 4 A schematic diagram of the business data processing device provided in this application is shown below. Figure 4 As shown, the processing device 40 provided in this embodiment includes:
[0120] Module 401 is used to acquire the finely tuned proprietary large model, business knowledge base, and business database.
[0121] The semantic understanding module 402 is used to obtain the data query question and generate a query semantic vector based on the data query question;
[0122] The semantic understanding module 402 is used to perform table data query operations in the business knowledge base based on the query semantic vector to obtain data table information;
[0123] The semantic understanding module 402 is also used to generate initial prompt words based on the data table information;
[0124] The query statement generation module 403 is used to generate a structured query statement to be executed based on the initial prompt words, data query questions, business database, and the fine-tuned proprietary big model.
[0125] The query module 404 is used to input the structured query statement to be executed into the preset executor. When the preset executor executes the structured query statement to be executed successfully, the query operation is performed from the business database to obtain preliminary result data.
[0126] The data presentation module 405 is used to process the preliminary result data to obtain the data to be presented.
[0127] The processing device 40 provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.
[0128] Figure 5 A schematic diagram of the structure of the electronic device provided in this application. Figure 5 As shown, the electronic device 50 provided in this embodiment includes at least one processor 501 and a memory 502. Optionally, the electronic device 50 further includes a communication component 503. The processor 501, memory 502, and communication component 503 are connected via a bus 504.
[0129] In a specific implementation, at least one processor 501 executes computer execution instructions stored in memory 502, causing at least one processor 501 to perform the above-described method.
[0130] The specific implementation process of processor 501 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.
[0131] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.
[0132] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.
[0133] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.
[0134] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.
[0135] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.
[0136] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.
[0137] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.
[0138] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.
[0139] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0140] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0141] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.
[0142] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.
Claims
1. A business data processing method, characterized in that, include: Obtain the finely tuned proprietary large model, business knowledge base, and business database; Obtain the data query question and generate a query semantic vector based on the data query question; Based on the query semantic vector, perform a table data query operation in the business knowledge base to obtain data table information; Based on the information in the data table, generate initial prompt words; Based on the initial prompt words, the data query question, the business database, and the finely tuned proprietary large model, a structured query statement to be executed is obtained; The structured query statement to be executed is input into a preset executor. When the preset executor executes the structured query statement to be executed successfully, a query operation is performed from the business database to obtain preliminary result data. The preliminary results data are processed to obtain the data to be presented.
2. The method according to claim 1, characterized in that, The acquisition of the finely tuned proprietary large model includes: Obtain the open-source basic large model and the public dataset of structured query statements, and perform initial fine-tuning on the open-source basic large model based on the public dataset of structured query statements to obtain the initial fine-tuned proprietary large model; The initial fine-tuned proprietary large model is then fine-tuned a second time based on a pre-built self-built business dataset to obtain the fine-tuned proprietary large model.
3. The method according to claim 1, characterized in that, Acquiring the business knowledge base includes: Obtain business data; The data table name, data table fields, data types, field types, field remarks, and field lengths are determined based on the business data. The data table name, data table fields, data types, field types, field remarks, and field lengths are used to generate a knowledge base table document as a business knowledge base according to a preset database table structure.
4. The method according to claim 1, characterized in that, The step of performing a table data query operation in the business knowledge base based on the query semantic vector to obtain data table information includes: Based on the query semantic vector, generate search suggestion terms; Based on the search suggestions and the preset general model, table data query operations are performed in the business knowledge base to obtain a paradigm output result containing the target table structure information from the business knowledge base; The output results of the above paradigm are used to perform data selection operations using a table name similarity hard matching algorithm to obtain data table information.
5. The method according to claim 1, characterized in that, The step of processing the preliminary result data to obtain the data to be presented includes: Based on the preliminary results data, a mapping and parsing process is performed to obtain key-value pair structure data; The structure data is parsed and concatenated based on the key values to obtain plain text markup language structure data.
6. The method according to claim 1, characterized in that, After inputting the structured query statement to be executed into the preset executor, the process further includes: When the execution result of the preset executor of the structured query statement to be executed fails, the structured query statement to be executed is re-inputted into the fine-tuned proprietary large model for adjustment processing to obtain the adjusted structured query statement to be executed. The adjusted structured query statement to be executed is then updated as the structured query statement to be executed, and the process jumps to the step of inputting the structured query statement to be executed into the preset executor.
7. The method according to any one of claims 1 to 6, characterized in that, After processing the preliminary result data to obtain the data to be presented, the process further includes: The query semantic vector is stored in a preset vector database as a historical query semantic vector, and the data to be presented is stored in a preset cache area as historical cache data. Obtain the new query question and generate a new query semantic vector based on the new query question; The new query semantic vector is used for matching in the preset vector database to obtain the matching result. When the matching result is detected to meet the preset matching success conditions, the historical cached data corresponding to the historical query semantic vector that was successfully matched is determined as the data to be presented now.
8. A business data processing device, characterized in that, include: The acquisition module is used to acquire the finely tuned proprietary large model, business knowledge base, and business database. The semantic understanding module is used to obtain the data query question and generate a query semantic vector based on the data query question; The semantic understanding module is used to perform table data query operations in the business knowledge base according to the query semantic vector to obtain data table information. The semantic understanding module is also used to generate initial prompt words based on the data table information; The query statement generation module is used to generate a structured query statement to be executed based on the initial prompt words, the data query question, the business database, and the fine-tuned proprietary big model. The query module is used to input the structured query statement to be executed into a preset executor. When the preset executor executes the structured query statement to be executed successfully, a query operation is performed from the business database to obtain preliminary result data. The data presentation module is used to process the preliminary result data to obtain the data to be presented.
9. An electronic device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 7.
Citation Information
Cited By
Information retrieval method and device
CN121365138A
Meteorological information network service operation index intelligent question-answering method and device
CN122332518A
