Data processing method and device based on visual database and electronic equipment

Abstract

The invention discloses a data processing method and device based on a visual database and electronic equipment, and relates to the field of data processing. The method comprises the following steps: acquiring a data processing request of a user; analyzing the data processing request to obtain a corresponding key field and a logic conjunction; according to the key field, performing semantic matching in a preset data model library to obtain a corresponding target data table and a target field; synthesizing the logic conjunction, the target data table and the target field to generate a corresponding database processing statement; executing the database processing statement, and obtaining a result data set generated after the database processing statement is executed; and transmitting the result data set to a visual database terminal, and displaying the result data set on the visual database terminal according to a preset visual template. By implementing the technical scheme provided by the invention, the data query efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a data processing method, device and electronic equipment based on a visual database. Background Art

[0002] With the rapid development of information technology, databases play a vital role in all walks of life. Enterprises and organizations rely on database systems to store, process and analyze large amounts of data to support key business activities such as decision making, customer management, and market analysis. As the basis for acquiring and processing this data, database query technology has become increasingly widely used. Almost every field, from finance, healthcare to public services, relies on this technology to optimize operations and enhance data insights.

[0003] At present, although database technology has made significant progress, there are still some technical challenges and operational limitations. First, traditional database queries require users to have specialized technical knowledge, such as SQL programming skills, which is a large threshold for users with non-technical backgrounds. Non-technical users may need more time to learn how to write effective query statements, or rely on others to write queries, which prolongs the time from demand to results and leads to low efficiency of database queries.

[0004] Therefore, there is an urgent need for a data processing method, device and electronic equipment based on a visual database. Summary of the invention

[0005] The present application provides a data processing method, device and electronic equipment based on a visual database, which improves the efficiency of data query.

[0006] In a first aspect of the present application, a data processing method based on a visual database is provided, the method comprising: obtaining a user's data processing request; parsing the data processing request to obtain corresponding key fields and logical connectives; performing semantic matching in a preset data model library according to the key fields to obtain corresponding target data tables and target fields, the preset data model library including multiple data tables, and the data tables including multiple fields; generating corresponding database processing statements based on the logical connectives, the target data tables, and the target fields; executing the database processing statements, and obtaining a result data set generated after executing the database processing statements; transmitting the result data set to a visual database terminal, and displaying the result data set on the visual database terminal according to a preset visualization template.

[0007] By adopting the above technical solution, by obtaining the user's data processing request, parsing the data processing request to obtain key fields and logical connectives, and performing semantic matching in the preset data model library, the corresponding target data table and target field are found, and then the database processing statement is generated and executed by integrating the logical connectives, target data table and field to obtain the result data set. Finally, the result data set is transmitted to the visual database terminal and displayed according to the preset visual template. This method converts the user's natural language request into a structured database operation, so that the user can easily obtain and analyze the required data without mastering the complex database language and model structure. At the same time, by displaying the result data set on the visual terminal, the data is presented in an intuitive form such as charts and reports, so that the user can quickly understand and gain insight into the data content, thereby improving the efficiency of data analysis and decision-making. This method simplifies the data processing process, lowers the user's usage threshold, improves the accessibility and value of the data, has significant practicality and convenience, and improves the efficiency and accuracy of data processing through automated processing and visual display methods.

[0008] Optionally, parsing the data processing request to obtain corresponding key fields and logical connectives specifically includes: using natural language processing technology to perform grammatical analysis on the data processing request to extract corresponding nouns, verbs and adjectives; matching the nouns, verbs and adjectives with a preset key field dictionary to obtain the key fields; and obtaining the logical connectives based on the logical connection relationship between the nouns, verbs and adjectives in the data processing request.

[0009] By adopting the above technical solution, when parsing data processing requests, natural language processing technology is used to perform grammatical analysis on the data processing requests, nouns, verbs and adjectives are extracted, and the key fields are obtained by matching with the preset key field dictionary. At the same time, logical connectives are obtained according to the logical connection relationship of these keywords in the data processing request. This parsing method makes full use of natural language processing technology, can accurately identify the semantic information in the user request, and extract key data elements and operation intentions. By matching with the preset key field dictionary, the user's expression can be mapped to the actual field of the database, eliminating the semantic gap between the user's expression and the database structure. At the same time, by analyzing the logical connection relationship between the keywords, the user's operation requirements for the data, such as conditional screening and association aggregation, can be inferred. This parsing method has a strong semantic understanding ability, can handle flexible and changeable user requests, and improves the accuracy and applicability of data processing.

[0010] Optionally, the matching of the nouns, the verbs and the adjectives with a preset key field dictionary to obtain the key field specifically includes: performing semantic similarity calculation in the preset key field dictionary based on the nouns, the verbs and the adjectives, the preset key field dictionary including multiple key field dictionary items; determining the key field dictionary item whose semantic similarity is greater than or equal to a preset similarity threshold as the key field.

[0011] By adopting the above technical solution, semantic similarity calculation is introduced when matching nouns, verbs and adjectives with preset key field dictionaries. By calculating the similarity between keywords and key field dictionary items, key field dictionary items with similarity greater than or equal to the preset similarity threshold are found as key fields. This matching method not only takes into account the literal form of keywords, but also their semantic connotations. It can handle synonyms and near-synonyms, and improves the generalization ability of key field recognition. Even if users use different expressions, they can find the corresponding key fields through semantic similarity matching to meet the diverse needs of users. At the same time, by setting a preset similarity threshold, the matching precision and recall rate can be controlled, and a balance can be struck between accuracy and completeness to obtain the best matching effect.

[0012] Optionally, the integration of the logical connectives, the target data table and the target fields to generate corresponding database processing statements specifically includes: determining the connection method between the multiple target fields according to the logical connectives, and generating a field connection expression according to the connection method; generating corresponding database query statements and database operation statements according to the table structure of the target data table, the field connection expression and a preset SQL template; combining the database query statement and the database operation statement to obtain the database processing statement, which includes query operations, insert operations, update operations and delete operations for the target data table.

[0013] By adopting the above technical solution, when generating database processing statements, first determine the connection method between target fields according to the logical connectives and generate field connection expressions. Then, according to the table structure of the target data table, the field connection expression and the preset SQL template, generate corresponding database query statements and operation statements, and combine them into complete database processing statements. This generation method encapsulates the underlying database technical details, and automatically generates database processing statements that conform to syntax and semantics through high-level logical expressions and SQL template mapping, reducing the workload of manual writing and debugging. At the same time, since the database processing statements are dynamically generated according to the actual table structure and fields, they can adapt to different database environments and demand changes, and have strong versatility and adaptability.

[0014] Optionally, the connection method between the multiple target fields is determined according to the logical connective to generate a field connection expression, which specifically includes: if the logical connective is and, the multiple target fields are inner-connected; if the logical connective is or, the multiple target fields are outer-connected; according to the inner connection and the outer connection, the field connection expression is generated using preset symbols.

[0015] By adopting the above technical solution, when generating a field connection expression, multiple target fields are connected by inner connection and outer connection respectively according to the logical connection words "and" and "or". This connection method matches the logical relationship in the user request and can accurately express the user's query intention. By adopting a preset template, the corresponding field connection expression can be automatically generated, avoiding the tediousness and errors of manually splicing connection conditions. This processing method makes reasonable use of the connection mechanism of the relational database. By associating multiple fields and tables, it realizes complex cross-table query and analysis, and expands the ability and scope of data processing. Users can flexibly combine various logical conditions in the request, which can be automatically converted into efficient and reliable database connection operations, improving the flexibility and scalability of data processing.

[0016] Optionally, after transmitting the result data set to the visual database terminal and displaying the result data set on the visual database terminal according to a preset visual template, the method further includes: setting interactive controls on the visual database terminal, the interactive controls including zoom controls, translation controls, rotation controls, filter controls and edit controls; when a user triggers the interactive control, obtaining an event type and parameters of the interactive control, and converting the event type and the parameters into corresponding data operation commands; according to the data operation command, calling a corresponding data processing interface to zoom, translate, rotate, filter or edit the result data set, and update the visual display in real time.

[0017] By adopting the above technical solution, multiple interactive controls are set on the visual database terminal, including zoom, pan, rotate, filter and edit controls, so that users can intuitively operate and analyze the visualized result data set. When the user triggers these controls, the event type and parameters of the interactive control are obtained and converted into corresponding data operation commands, such as zoom command, pan command, and filter command. Then, according to these commands, the corresponding data processing interface is called to perform real-time zoom, pan, rotate, filter or edit the result data set, and synchronously update the visual view. Through graphical operations, users can quickly zoom in on the data area of ​​interest, change different perspectives and layouts, add or remove data subsets, and modify key data points, so as to obtain a personalized analysis experience. This visual interaction mechanism enables users to actively participate in the data exploration process, deepen their understanding and application of data, and give full play to the value of data.

[0018] Optionally, before obtaining the user's data processing request, the method also includes: obtaining structured data from multiple data sources, the structured data including data table identifiers and data table structures; identifying associations between different data tables in the multiple data sources, and constructing a data table association model; based on the data table identifiers and the data table structures, extracting data table names and field information, and establishing data table indexes; and constructing the preset data model library based on the data table association model and the data table indexes.

[0019] By adopting the above technical solution, the structured data of multiple heterogeneous data sources are preprocessed and modeled before obtaining user requests. First, metadata information such as data table identifiers and table structures of each data source is extracted. Then, by identifying the association relationship between different data tables, a global data table association model is constructed to characterize the semantic connection between data sources. Next, based on the data table identifier and table structure, detailed table names and field information are further extracted, and data table indexes are established to facilitate rapid retrieval and location of data. Finally, the data table association model and data table index are organized to build a preset data model library as a unified knowledge source for processing user requests. This preprocessing mechanism eliminates the structural and semantic differences between data sources by analyzing and integrating multi-source heterogeneous data in advance, and provides a unified data foundation for parsing and processing user requests. Users do not need to understand the complexity and differences of the underlying data sources, but only need to query and analyze based on the association model and index, which simplifies the complexity of data use. At the same time, since the preset data model library has been preprocessed and optimized, it has high retrieval and computing efficiency, can support fast data response and real-time analysis, and improve the system performance and user experience.

[0020] In a second aspect of the present application, a data processing device based on a visual database is provided, the device comprising: an acquisition module and a processing module, wherein: the acquisition module is used to acquire a user's data processing request; the processing module is used to parse the data processing request to obtain corresponding key fields and logical connectives; the processing module is also used to perform semantic matching in a preset data model library according to the key fields to obtain corresponding target data tables and target fields; the processing module is also used to integrate the logical connectives, the target data tables and the target fields to generate corresponding database processing statements; the processing module is also used to execute the database processing statements and acquire a result data set corresponding to the database processing statements; the processing module is also used to transmit the result data set to a visual database terminal, and display the result data set on the visual database terminal according to a preset visualization template.

[0021] In the third aspect of the present application, an electronic device is provided, including a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes any one of the methods described above.

[0022] In a fourth aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores instructions, and when the instructions are executed, any of the methods described above is executed.

[0023] In summary, one or more technical solutions provided by this application have at least the following technical effects or advantages: 1. Obtain the user's data processing request, parse the data processing request to obtain key fields and logical connectives, and perform semantic matching in the preset data model library to find the corresponding target data table and target field, and then generate and execute the database processing statement based on the logical connectives, target data table and field to obtain the result data set. Finally, the result data set is transmitted to the visual database terminal and displayed according to the preset visualization template. This method converts the user's natural language request into a structured database operation, so that the user can easily obtain and analyze the required data without mastering complex database language and model structure. At the same time, by displaying the result data set on the visualization terminal and presenting the data in intuitive forms such as charts and reports, users can quickly understand and gain insight into the data content, thereby improving the efficiency of data analysis and decision-making. This method simplifies the data processing process, lowers the user's usage threshold, improves the accessibility and value of data, and has significant practicality and convenience.

[0024] 2. When parsing data processing requests, natural language processing technology is used to perform grammatical analysis on data processing requests, extract nouns, verbs and adjectives, and match them with the preset key field dictionary to obtain key fields. At the same time, logical connectives are obtained based on the logical connection relationship of these keywords in the data processing request. This parsing method makes full use of natural language processing technology, can accurately identify the semantic information in user requests, and extract key data elements and operation intentions. By matching with the preset key field dictionary, the user's expression can be mapped to the actual field of the database, eliminating the semantic gap between the user's expression and the database structure. At the same time, by analyzing the logical connection relationship between keywords, the user's operational requirements for data, such as conditional screening and association aggregation, can be inferred. This parsing method has a strong semantic understanding ability, can handle flexible and changeable user requests, and improves the accuracy and applicability of data processing.

[0025] 3. When matching nouns, verbs, and adjectives with the preset key field dictionary, semantic similarity calculation is introduced. By calculating the similarity between the keywords and the key field dictionary items, the key field dictionary items with a similarity greater than or equal to the preset similarity threshold are found as key fields. This matching method not only takes into account the literal form of the keywords, but also their semantic connotations. It can handle synonyms and near-synonyms, and improves the generalization ability of key field recognition. Even if the user uses different expressions, the corresponding key fields can be found through semantic similarity matching to meet the diverse needs of users. At the same time, by setting the preset similarity threshold, the matching precision and recall rate can be controlled, and a balance can be struck between accuracy and completeness to obtain the best matching effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a flowchart of a data processing method based on a visual database disclosed in an embodiment of the present application; Figure 2 It is a module schematic diagram of a data processing device based on a visual database disclosed in an embodiment of the present application; Figure 3 It is a structural schematic diagram of an electronic device disclosed in an embodiment of the present application.

[0027] Explanation of the reference numerals: 201, acquisition module; 202, processing module; 300, electronic device; 301, processor; 302, communication bus; 303, user interface; 304, network interface; 305, memory. DETAILED DESCRIPTION

[0028] In order to enable technicians in this field to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.

[0029] In the description of the embodiments of the present application, words such as "for example" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "for example" or "for example" is intended to present related concepts in a specific way.

[0030] In the description of the embodiments of the present application, the meaning of the term "multiple" refers to two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprise", "have" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0031] This application provides a data processing method based on a visual database, referring to Figure 1 , Figure 1 1 is a flow chart of a data processing method based on a visual database provided in an embodiment of the present application. The method is applied to a server, which is a server that executes a data processing program based on a visual database. The server can be a single server, a server cluster composed of multiple servers, or a cloud computing service center. The method includes steps S101 to S106, which are as follows: Step S101: Obtain the user's data processing request.

[0032] In a possible implementation, before step S101, the method further includes: acquiring structured data from multiple data sources, the structured data including data table identifiers and data table structures; identifying associations between different data tables in the multiple data sources, and constructing a data table association model; extracting data table names and field information based on the data table identifiers and data table structures, and establishing data table indexes; and constructing a preset data model library based on the data table association models and data table indexes.

[0033] Specifically, the server obtains structured data from multiple heterogeneous data sources. These data sources may come from different business systems, such as sales systems, warehousing systems, and financial systems. Each data source usually contains multiple data tables, and each data table is described by a table identifier (such as table name) and a table structure (such as field name, field type, constraints, etc.). The server establishes a connection with the data source, such as a JDBC connection, and obtains the identifier and structure information of these data tables through metadata queries. Next, the server identifies the association between different data sources and different data tables and builds a global data table association model. The purpose of this step is to reveal the semantic connection between data tables and provide a basis for subsequent cross-table association queries. The server identifies the table association relationship through explicit foreign key associations, and directly obtains the primary and foreign key relationship between the two tables by analyzing the foreign key constraints of the data table. For example, the customer_id field of the order table refers to the id primary key of the customer table, so there is an explicit association between the order table and the customer table. The data table association model is represented in the form of a directed graph. The nodes in the graph represent data tables, the edges represent the association relationship between tables, and the direction of the edge represents the direction of the association reference.

[0034] After building the data table association model, the server extracts the table name and field information based on the data table identifier and table structure, and builds a global data table index. The data table index is similar to a data dictionary, which records the detailed meta information of each table and each field, such as table name, field name, field data type, field comment, etc. This meta information is used for subsequent keyword matching on the one hand, and also provides necessary information for automatically generating database operation statements on the other hand. Finally, the server integrates the data table association model and the data table index to build a preset data model library.

[0035] In step S101, the server is responsible for receiving and obtaining the data processing request submitted by the user. The user usually sends the data processing request to the server in natural language or structured form through a client application or a web browser. The server needs to provide corresponding interfaces and protocols to interact and communicate with the client and obtain the content of the data processing request.

[0036] Step S102: Parse the data processing request to obtain corresponding key fields and logical connectives.

[0037] In step S102, the data processing request is parsed to obtain corresponding key fields and logical connectives, specifically including: using natural language processing technology to perform grammatical analysis on the data processing request to extract corresponding nouns, verbs and adjectives; matching the nouns, verbs and adjectives with a preset key field dictionary to obtain key fields; and obtaining logical connectives based on the logical connection relationship between nouns, verbs and adjectives in the data processing request.

[0038] Specifically, the server first inputs the user's data processing request into an NLP pipeline for processing. In the NLP pipeline, word segmentation and part-of-speech tagging are first performed to divide the data processing request text into independent vocabulary units, and each vocabulary is tagged with part-of-speech to determine whether it is a noun, verb, adjective or other part of speech. Then the server identifies important entity objects from the data processing request text, such as names of people, places, names of organizations, dates, numbers, etc. This step can help the server understand the key business objects involved in the request. Finally, the server analyzes the dependency relationships between words in the data processing request text, such as subject-predicate relationships, verb-object relationships, and attributive-predicate relationships. This step can help the server understand the semantic structure and logical relationships in the request.

[0039] After the NLP pipeline processes the data processing request text, the server will obtain rich semantic information, including parts of speech, entities, dependencies, etc. Then, the server extracts key fields and logical connectors from this information. For the extraction of key fields, the server focuses on content words such as nouns, verbs, and adjectives in the request text, and matches them with the preset key field dictionary. The preset key field dictionary is a predefined mapping table that maps common business fields to actual fields in the database. For example, the dictionary may define "sales" corresponding to the "sales_amount" field in the database, "goods" corresponding to the "product" table, and so on. Through dictionary matching, the server can convert the natural language fields in the request into structured fields in the database.

[0040] For the extraction of logical connectives, the server mainly analyzes the logical relationship between words in the request text, such as "and", "or", "greater than", "less than", "equal to", "sort", "group", etc. These logical relationships are usually reflected in the dependency syntactic structure of the request, and the server can identify logical connectives by analyzing the dependency relationship. For example, in "the 10 products with the highest sales", "highest" and "sales" are in a subject-predicate relationship, and the server can identify it as a "sort" condition; in "products with sales greater than 1,000", "greater than" and "sales" are in a subject-predicate relationship, and the server can identify it as a "comparison" condition.

[0041] In one possible implementation, nouns, verbs, and adjectives are matched with a preset key field dictionary to obtain key fields, specifically including: based on nouns, verbs, and adjectives, semantic similarity is calculated in the preset key field dictionary, the preset key field dictionary includes multiple key field dictionary items; and key field dictionary items whose semantic similarity is greater than or equal to a preset similarity threshold are determined as key fields.

[0042] Specifically, the server first loads the preset key field dictionary. The preset key field dictionary is a collection of key fields, each of which corresponds to an actual field in the database. For example, the dictionary may contain the following key field items: "sales amount" corresponds to the "sales_amount" field in the database; "sales volume": corresponds to the "sales_volume" field in the database; "product name": corresponds to the "product_name" field in the database; "product price": corresponds to the "product_price" field in the database; "order date": corresponds to the "order_date" field in the database.

[0043] Next, the server calculates the semantic similarity between the keywords such as nouns, verbs, and adjectives extracted from the data processing request text and each key field dictionary item in the key field dictionary. The server uses cosine similarity based on word vectors to convert both keywords and key field dictionary items into word vector representations, and calculates the cosine angle between the two word vectors as the similarity. The server will calculate a semantic similarity score for each keyword and each key field item. Then, the server will set a preset similarity threshold (such as 0.8) and identify key field items with similarity scores greater than or equal to the preset similarity threshold as fields that match the key field dictionary items.

[0044] For example, for the request text "Query the 10 products with the highest sales last year", the keywords extracted by the server include "sales" and "products". In the preset key field dictionary, the keyword "sales" may have a high similarity with the "sales_amount" field (such as 0.9), but a low similarity with other fields (such as the "product_name" field has a similarity of only 0.2). Therefore, the server will identify the "sales_amount" field as a key field that matches "sales". Similarly, the keyword "product" may have the highest similarity with the "product_name" field, and the server will also identify it as a matching key field.

[0045] Step S103: According to the key field, semantic matching is performed in a preset data model library to obtain a corresponding target data table and target field, wherein the preset data model library includes a plurality of data tables, and the data table includes a plurality of fields.

[0046] In step S103, the server loads the preset data model library. The preset data model library is automatically constructed when the system is initialized or the data source is connected, and contains the meta information of all data tables and fields included in the data table in the data source, as well as the relationship between them. Next, the server will extract the key fields from the data processing request text and perform semantic matching with the tables and fields in the preset data model library. The matching process can be divided into two levels: First, table-level matching. The server first determines which data table the key field belongs to. The server infers based on the semantic type and contextual information of the key field. For example, the key field "sales" generally appears in tables related to "sales" or "orders", and "product name" generally appears in tables related to "products" or "products". The server calculates the semantic relevance of the key field and each data table, and selects the table with the highest relevance as the target data table.

[0047] The second is field-level matching. After the server determines the target data table, it further determines which specific field in the target data table the key field corresponds to. This step mainly uses the semantic similarity of field names, as well as meta-information such as field data types and constraints. For example, the key field "sales" may have the highest semantic similarity with the "sales_amount" field in the "sales" table, and the data types also match (both are numeric), so the server determines the "sales_amount" field as the target field. Through semantic matching at the table level and field level, the server can eventually map the key fields to the target data table and target fields in the preset data model library. These mapping relationships provide the necessary information for subsequent SQL query generation.

[0048] Step S104: Integrate the logical connectives, target data table and target field to generate corresponding database processing statements.

[0049] In step S104, logical connectives, target data tables and target fields are integrated to generate corresponding database processing statements, specifically including: determining the connection method between multiple target fields according to the logical connectives, and generating field connection expressions according to the connection method; generating corresponding database query statements and database operation statements according to the table structure, field connection expressions and preset SQL templates of the target data table; combining the database query statements and database operation statements to obtain database processing statements, which include query operations, insert operations, update operations and delete operations for the target data table.

[0050] Specifically, the server automatically generates corresponding database processing statements by comprehensively utilizing the previously extracted logical connectives, target data tables, target fields, and other information. Database processing statements are expressed in the form of SQL (Structured Query Language), including common operations such as query, insert, update, and delete. The goal of this step is to convert natural language requests into structured statements that can be directly executed by the database, achieving seamless integration of human-computer interaction and data operations.

[0051] Specifically, the server first determines the connection method between multiple target fields based on the logical connectives and generates a field connection expression. Logical connectives reflect the logical relationship between different conditions or operations in the request, such as "AND", "OR", "NOT", etc. The field connection expression applies these logical relationships to specific fields to form a composite condition. For example, in the request "Query orders with sales greater than 1,000 yuan and sales date in 2022", "sales amount" and "sales date" are two target fields, "greater than" and "in" are conditional operators, and "and" is a logical connective. The corresponding field connection expression can be written as "sales_amount>1000 AND sales_date BETWEEN '2022-01-01'AND '2022-12-31'".

[0052] Next, the server generates the corresponding database query statements and database operation statements based on the table structure, field connection expressions, and preset SQL templates of the target data table. The table structure defines the metadata of the data table, such as the field name, data type, and constraints, and is the basis for constructing SQL statements. The preset SQL template provides the commonly used SQL statement structure and format, which can be filled and replaced according to the specific table name, field name, conditional expression and other parameters to generate a complete SQL statement. For data operations such as insert, update, and delete, the server also has corresponding preset SQL templates. The server can select the appropriate preset SQL template and fill in the parameters according to the semantics of the request and the extracted information to generate the corresponding data operation statements.

[0053] Finally, the server combines the generated database query statements and database operation statements to form a complete database processing statement. A database processing statement is an independent query or operation statement, or it may be a transaction or batch consisting of multiple statements.

[0054] In a possible implementation, the connection method between multiple target fields is determined according to a logical connective, and a field connection expression is generated, specifically including: if the logical connective is and, the multiple target fields are inner connected; if the logical connective is or, the multiple target fields are outer connected; according to the inner connection and the outer connection, the field connection expression is generated using preset symbols.

[0055] Specifically, the server automatically determines the connection method between multiple target fields based on the logical connectives in the request and generates the corresponding field connection expressions. There are usually two types of connection methods: inner connection and outer connection, which correspond to different syntax and semantics in SQL statements. The server needs to select the appropriate connection method based on the type of logical connectives and convert it into a field connection expression in SQL.

[0056] Specifically, when the logical connective is "and", the server will perform inner joins on multiple target fields. Inner join is a common join method used to query records in multiple tables that simultaneously meet certain conditions. Its characteristic is that it only returns records with the same value on the join field, and unmatched records are filtered out. When the logical connective is "or (OR)", the server will perform outer joins on multiple target fields. Outer join is an extended join method used to query all records in a table and matching records in another table. Its characteristic is that it returns not only matching records but also unmatched records, and unmatched fields are filled with NULL. Finally, the server fills the generated field join expression into the preset SQL template, and combines it with other query conditions, target fields, etc. to form a complete database processing statement.

[0057] Step S105: Execute the database processing statement, and obtain a result data set generated after executing the database processing statement.

[0058] In step S105, the server first establishes a connection with the database. The server specifies the type of database (such as MySQL, Oracle, SQL Server, etc.), host address, port number, database name, user name and password, etc. This information can be configured when the system is initialized, or it can be dynamically obtained according to user requests. After the connection is established, the server can interact with the database, execute database processing statements and obtain return results.

[0059] After the connection is established, the server sends the database processing statement generated in the previous step to the database for execution. After receiving the SQL statement, the database will perform corresponding data operations on it. Taking the query statement as an example, the database will scan the relevant tables and indexes according to the query conditions, extract the records that meet the conditions, and calculate the results according to the query fields and aggregate functions. After the database executes the SQL statement, it will return the query results to the server in the form of a result data set. The result data set is a two-dimensional table structure that contains multiple rows and columns of data. Each column corresponds to a target field in the query statement, and each row corresponds to a record that meets the query conditions.

[0060] In addition to query statements, the server can also execute data operation statements such as insert, update, and delete. These statements do not return result data sets, but will affect the status of the data in the database. The server usually returns the corresponding status code and the number of affected rows based on the execution results of these statements so that users can understand the effect of the operation.

[0061] Step S106: Transmit the result data set to the visual database terminal, and display the result data set on the visual database terminal according to a preset visual template.

[0062] In step S106, the server transmits the result data set to the visual database terminal. The visual database terminal is usually a web application or desktop client with rich data display and interactive functions. Data transmission between the server and the visual database terminal can be achieved in various ways, such as HTTP interface, WebSocket, etc. The transmitted data format can be JSON, XML, binary, etc., depending on the requirements of the visual database terminal and the design of the system. The server serializes the result data set into a corresponding format and sends it to the visual database terminal through a network protocol.

[0063] Next, the server displays the result dataset on the terminal according to the preset visualization template. The visualization template defines how the data is displayed and interacted with on the visualization database terminal, such as tables, line charts, pie charts, maps, etc. For example, the server can define a table template for a result dataset in JSON format, specifying table attributes such as column width, alignment, sorting, and cell formatting rules, link jumps, and other interactive methods. When a user requests to view the result dataset in a table, the server loads the template, fills the JSON data into each cell of the table template, and renders the complete table on the terminal. Users can sort, filter, and page the table, or click a cell to jump to the details page.

[0064] In addition to tables, the server can also define other types of visualization templates for result data sets, such as line chart templates, pie chart templates, etc. These templates can select appropriate visual elements and layout methods based on the characteristics of the data and user needs to display data in the most intuitive and effective way. For example, for a result data set containing time series, the server can use a line chart template, with the time field as the X-axis and the value field as the Y-axis, to display the trend and change of the data. For a result data set containing classified summaries, the server can use a pie chart template, with the classification field as the sector and the value field as the size of the sector, to display the proportion and percentage of the data.

[0065] In a possible implementation, after step S106, the method further includes: setting interactive controls on the visual database terminal, the interactive controls including zoom controls, translation controls, rotation controls, filter controls, and edit controls; when a user triggers the interactive controls, obtaining event types and parameters of the interactive controls, and converting the event types and parameters into corresponding data operation commands; according to the data operation commands, calling corresponding data processing interfaces, zooming, translating, rotating, filtering, or editing the result data set, and updating the visual display in real time.

[0066] Specifically, the server first sets up interactive controls on the visual database terminal. Interactive controls can be common UI components such as buttons, sliders, drop-down boxes, or customized graphic elements or gestures. Each control has specific event types and parameters to describe the user's operation intentions and specific content. For example, a zoom control can be a slider, and users can drag the slider to zoom in or out of the visualization results; a pan control can be four directional arrows, and users can click the arrows to move the visualization view up, down, left, and right; a filter control can be a drop-down box, and users can select specific conditions to filter part of the data in the result data set.

[0067] When users trigger these interactive controls, the visual database terminal captures the corresponding events and sends the event type and parameters to the server. After receiving these event information, the server converts them into corresponding data operation commands. Data operation commands are a standardized format that describes the specific requirements for what kind of processing to do with the result data set. For example, a zoom event can be converted into a command to set the zoom ratio, a pan event can be converted into a command to set the data offset, and a filter event can be converted into a command to set the data filter condition.

[0068] The server calls the corresponding data processing interface according to the data operation command and processes the result data set accordingly. The data processing interface is a set of functions or methods pre-defined by the server to perform common data operations, such as zooming, panning, rotating, filtering, editing, etc. These interfaces accept standardized data operation commands as parameters, process the result data set according to the command requirements, and return the processed data results.

[0069] Taking the zoom operation as an example, the server can define a zoom interface that accepts the zoom ratio as a parameter. When the user triggers the zoom control and drags the slider to a certain position, the visual database terminal captures the zoom event and sends the current slider position as a parameter to the server. The server converts the slider position into a zoom ratio command, such as "setZoom(1.5)", which means to enlarge the visualization result by 1.5 times. The server then calls the zoom interface and multiplies all the values ​​in the result data set by 1.5 to obtain the enlarged new data set. Finally, the server returns the new data set to the terminal, updates the visualization view, and presents the enlarged effect.

[0070] Similarly, for panning operations, the server can define a panning interface that accepts horizontal and vertical offsets as parameters. When the user triggers the panning control, such as clicking the right arrow, the terminal captures the panning event and sends the offset (such as "10px") to the server. The server converts the offset into a panning command, such as "setOffset(10, 0)", which means to move the view horizontally to the right by 10 pixels. The server then calls the panning interface and adds the offset to all coordinate values ​​in the result data set to obtain the new coordinates after the panning. Finally, the server returns the new coordinates to the visual database terminal, updates the visual view, and presents the panning effect.

[0071] For filtering operations, the server can define a filtering interface that accepts one or more filtering conditions as parameters. When the user selects a specific condition in the filtering control, such as "sales greater than 10,000", the terminal captures the filtering event and sends the condition to the server. The server converts the condition into a filtering command, such as "setFilter('total_amount>10000')", which means filtering out data with sales greater than 10,000 from the result data set. The server then calls the filtering interface to filter the result data set to obtain a subset that meets the conditions. Finally, the server returns the subset to the terminal and updates the visualization view to only display the filtered data.

[0072] For editing operations, the server can define an editing interface that accepts the modified data value as a parameter. When the user modifies the value of a data point in the edit control, such as changing a customer's sales from 5,000 to 6,000, the terminal captures the edit event and sends the new value to the server. The server converts the new value into an edit command, such as "setData(1, 'total_amount', 6000)", which means updating the sales field value of the first row to 6,000. The server then calls the editing interface to modify the result data set to obtain the updated data. Finally, the server returns the updated data to the visual database terminal, updates the visual view, and presents the edited effect.

[0073] Reference Figure 2 The present application also provides a data processing device based on a visual database, which is a server. The server includes an acquisition module 201 and a processing module 202, wherein: the acquisition module 201 is used to acquire the user's data processing request; the processing module 202 is used to parse the data processing request to obtain the corresponding key fields and logical connectives; the processing module 202 is also used to perform semantic matching in a preset data model library according to the key fields to obtain the corresponding target data table and target field, and the preset data model library includes multiple data tables, and the data table includes multiple fields; the processing module 202 is also used to synthesize the logical connectives, the target data table and the target field to generate the corresponding database processing statement; the processing module 202 is also used to execute the database processing statement and obtain the result data set generated after executing the database processing statement; the processing module 202 is also used to transmit the result data set to the visual database terminal, and display the result data set on the visual database terminal according to the preset visualization template.

[0074] In a possible implementation, the processing module 202 parses the data processing request to obtain corresponding key fields and logical connectives, specifically including: the processing module 202 uses natural language processing technology to perform grammatical analysis on the data processing request to extract corresponding nouns, verbs and adjectives; the processing module 202 matches the nouns, verbs and adjectives with a preset key field dictionary to obtain key fields; the processing module 202 obtains logical connectives based on the logical connection relationship between the nouns, verbs and adjectives in the data processing request.

[0075] In one possible implementation, the processing module 202 matches nouns, verbs and adjectives with a preset key field dictionary to obtain key fields, specifically including: the processing module 202 calculates semantic similarity in the preset key field dictionary based on nouns, verbs and adjectives, and the preset key field dictionary includes multiple key field dictionary items; the processing module 202 determines that the key field dictionary item whose semantic similarity is greater than or equal to the preset similarity threshold is the key field.

[0076] In a possible implementation, the processing module 202 integrates logical connectives, target data tables, and target fields to generate corresponding database processing statements, specifically including: the processing module 202 determines the connection method between multiple target fields according to the logical connectives, and generates a field connection expression according to the connection method; the processing module 202 generates corresponding database query statements and database operation statements according to the table structure of the target data table, the field connection expression, and a preset SQL template; the processing module 202 combines the database query statement and the database operation statement to obtain a database processing statement, and the database processing statement includes a query operation, an insert operation, an update operation, and a delete operation for the target data table.

[0077] In a possible implementation, the processing module 202 determines the connection method between multiple target fields according to the logical connective and generates a field connection expression, specifically including: if the logical connective is and, the processing module 202 performs an inner connection on the multiple target fields; if the logical connective is or, the processing module 202 performs an outer connection on the multiple target fields; the processing module 202 generates a field connection expression based on the inner connection and the outer connection using preset symbols.

[0078] In a possible implementation, after the processing module 202 transmits the result data set to the visual database terminal and displays the result data set on the visual database terminal according to a preset visualization template, the method further includes: the processing module 202 sets interactive controls on the visual database terminal, and the interactive controls include zoom controls, translation controls, rotation controls, filter controls, and edit controls; when the user triggers the interactive control, the processing module 202 obtains the event type and parameters of the interactive control, and converts the event type and parameters into corresponding data operation commands; the processing module 202 calls the corresponding data processing interface according to the data operation command, zooms, translates, rotates, filters or edits the result data set, and updates the visualization display in real time.

[0079] In a possible implementation, before the acquisition module 201 acquires the user's data processing request, the method further includes: the acquisition module 201 acquires structured data from multiple data sources, the structured data including data table identifiers and data table structures; the processing module 202 identifies the associations between different data tables in the multiple data sources, and constructs a data table association model; the processing module 202 extracts the data table name and field information based on the data table identifier and the data table structure, and establishes a data table index; the processing module 202 constructs a preset data model library based on the data table association model and the data table index.

[0080] It should be noted that: when the device provided in the above embodiment realizes its function, only the division of the above functional modules is used as an example. In actual application, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0081] The present application also provides an electronic device. Figure 3 , Figure 3 The electronic device 300 may include: at least one processor 301 , at least one network interface 304 , a user interface 303 , a memory 305 , and at least one communication bus 302 .

[0082] The communication bus 302 is used to realize the connection and communication between these components.

[0083] The user interface 303 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 303 may also include a standard wired interface and a wireless interface.

[0084] The network interface 304 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0085] Among them, the processor 301 may include one or more processing cores. The processor 301 uses various interfaces and lines to connect various parts in the entire server, and executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 305, and calling data stored in the memory 305. Optionally, the processor 301 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 301 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 301, and it can be implemented separately through a chip.

[0086] Among them, the memory 305 may include a random access memory (Random Access Memory, RAM) and may also include a read-only memory (Read-Only Memory). Optionally, the memory 305 includes a non-transitory computer-readable storage medium. The memory 305 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 305 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 305 may optionally also be at least one storage device located away from the aforementioned processor 301. Refer to Figure 3 , the memory 305 as a computer storage medium may include an operating system, a network communication module, a user interface module and an application program of a data processing method based on a visual database.

[0087] exist Figure 3In the electronic device 300 shown, the user interface 303 is mainly used to provide an input interface for the user and obtain the data input by the user; and the processor 301 can be used to call the application program stored in the memory 305 for a data processing method based on a visual database. When executed by one or more processors 301, the electronic device 300 executes one or more of the methods described in the above embodiments. It should be noted that for the aforementioned method embodiments, for the sake of simple description, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for the present application.

[0088] The present application also provides a computer-readable storage medium, which stores instructions. When executed by one or more processors 301, the electronic device 300 executes one or more of the methods described in the above embodiments.

[0089] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0090] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0091] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0092] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0093] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: various media that can store program codes, such as USB flash drives, mobile hard drives, magnetic disks or optical disks.

[0094] The above is only an exemplary embodiment of the present disclosure and cannot be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the disclosure of the specification and the truth of practice, those skilled in the art will easily think of other embodiments of the present disclosure.

[0095] This application is intended to cover any variation, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the art not described in the present disclosure. The description and examples are to be regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A data processing method based on a visual database, characterized in that: The method comprises: Get the user's data processing request; Parsing the data processing request to obtain corresponding key fields and logical connectives; According to the key field, semantic matching is performed in a preset data model library to obtain a corresponding target data table and target field, wherein the preset data model library includes multiple data tables, and the data table includes multiple fields; Combining the logical connectives, the target data table and the target field, generating a corresponding database processing statement; Executing the database processing statement, and obtaining a result data set generated after executing the database processing statement; The result data set is transmitted to a visual database terminal, and the result data set is displayed on the visual database terminal according to a preset visual template.

2. The method according to claim 1, characterized in that The parsing of the data processing request to obtain corresponding key fields and logical connectives specifically includes: Using natural language processing technology to perform grammatical analysis on the data processing request to extract corresponding nouns, verbs and adjectives; Matching the noun, the verb and the adjective with a preset key field dictionary to obtain the key field; The logical connective is obtained according to the logical connection relationship between the noun, the verb and the adjective in the data processing request.

3. The method according to claim 2, characterized in that The matching of the noun, the verb and the adjective with a preset key field dictionary to obtain the key field specifically includes: Based on the noun, the verb and the adjective, a semantic similarity calculation is performed in the preset key field dictionary, where the preset key field dictionary includes a plurality of key field dictionary items; Determine the key field dictionary item whose semantic similarity is greater than or equal to a preset similarity threshold as the key field.

4. The method according to claim 1, characterized in that: The step of synthesizing the logical connectives, the target data table, and the target field to generate a corresponding database processing statement specifically includes: Determine a connection mode between the plurality of target fields according to the logical connectives, and generate a field connection expression according to the connection mode; Generate corresponding database query statements and database operation statements according to the table structure of the target data table, the field connection expression and the preset SQL template; The database query statement and the database operation statement are combined to obtain the database processing statement, which includes a query operation, an insert operation, an update operation, and a delete operation for the target data table.

5. The method according to claim 4, characterized in that Determining the connection mode between the plurality of target fields according to the logical connectives and generating a field connection expression specifically includes: If the logical connective is and, then multiple target fields are inner-connected; If the logical connective is or, then multiple target fields are outer-connected; The field connection expression is generated using preset symbols according to the inner connection and the outer connection.

6. The method according to claim 1, characterized in that After transmitting the result data set to the visual database terminal and displaying the result data set on the visual database terminal according to a preset visual template, the method further includes: An interactive control is provided on the visual database terminal, wherein the interactive control includes a zoom control, a translation control, a rotation control, a filter control, and an edit control; When the user triggers the interactive control, the event type and parameters of the interactive control are obtained, and the event type and the parameters are converted into corresponding data operation commands; According to the data operation command, the corresponding data processing interface is called to zoom, translate, rotate, filter or edit the result data set, and update the visual display in real time.

7. The method according to claim 1, characterized in that Before obtaining the user's data processing request, the method further includes: Acquire structured data from multiple data sources, wherein the structured data includes a data table identifier and a data table structure; Identify the association relationship between different data tables in the multiple data sources and build a data table association model; Based on the data table identifier and the data table structure, extract the data table name and field information, and establish a data table index; The preset data model library is constructed according to the data table association model and the data table index.

8. A data processing device based on a visual database, characterized in that: The device comprises an acquisition module (201) and a processing module (202), wherein: The acquisition module (201) is used to acquire a user's data processing request; The processing module (202) is used to parse the data processing request to obtain corresponding key fields and logical connectives; The processing module (202) is further used to perform semantic matching in a preset data model library according to the key field to obtain a corresponding target data table and target field, wherein the preset data model library includes a plurality of data tables, and the data table includes a plurality of fields; The processing module (202) is further used to synthesize the logical connectives, the target data table and the target field to generate a corresponding database processing statement; The processing module (202) is further used to execute the database processing statement and obtain a result data set generated after executing the database processing statement; The processing module (202) is further used to transmit the result data set to a visual database terminal, and display the result data set on the visual database terminal according to a preset visual template.

9. An electronic device, characterized in that: The electronic device (300) comprises a processor (301), a memory (305), a user interface (303) and a network interface (304), wherein the memory (305) is used to store instructions, the user interface (303) and the network interface (304) are used to communicate with other devices, and the processor (301) is used to execute the instructions stored in the memory (305) so that the electronic device (300) executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed, the method according to any one of claims 1 to 7 is executed.

Images