Data analysis method and system, electronic equipment and storage medium

Abstract

The invention provides a data analysis method and system, electronic equipment and a storage medium, and the method comprises the steps: obtaining a data query natural language inputted by a user, carrying out the intention recognition of the data query natural language, and obtaining a target query intention and a query vector corresponding to the target query intention; recalling multiple pieces of candidate data set information from multiple pieces of preset data set information according to the similarity between the query vector and multiple preset data set vectors, and performing matching degree scoring on the candidate data set information according to the target query intention to obtain selected data set information, generating a structured semantic query language based on the target query intention and the selected data set information, and converting the semantic query language into a physical query language of a corresponding database according to a preset mapping relationship so as to query the data set from the database to obtain to-be-analyzed data, the preset mapping relationship represents a mapping relationship between the semantic logic layer and the business physical layer; according to the method, the data set and data source expansion can be accurately positioned, and the query generation efficiency is improved.

Description

Technical Field

[0001] This invention relates to the field of data query technology, and in particular to a data analysis method, system, electronic device, and storage medium. Background Technology

[0002] In the process of enterprise digital transformation, data analytics has become an important support for business decision-making. Data analytics systems typically require users to master professional technologies such as Structured Query Language (SQL), which creates a technical barrier.

[0003] Natural language query systems (NLPs) in related technologies still suffer from numerous architectural design problems in practical applications. First, semantic understanding is not precise enough. They struggle to accurately understand users' natural language query intent, often resulting in irrelevant answers. This is particularly challenging in scenarios with large enterprise datasets, where accurately locating the specific dataset required by the user is a significant challenge. Second, query generation efficiency is low. The conversion process from natural language to SQL lacks systematic architectural support, leading to inconsistent SQL quality and low execution efficiency. Third, system integration is difficult. Often employing closed architectures, they struggle to integrate with existing enterprise data infrastructure and adapt to new data sources and analytical needs. Furthermore, there is a lack of semantic layer abstraction. Directly mapping natural language to physical table structures ignores the importance of the business semantic layer, making the system difficult to understand and maintain. Finally, scalability is insufficient. Traditional architectures struggle to support new artificial intelligence (AI) models, new data source types, and new visualization methods, limiting system development.

[0004] Therefore, how to design a conversational data analysis system architecture that is accurate in semantic understanding, efficient in query generation, and easy to integrate and extend has become an urgent technical problem to be solved. Summary of the Invention

[0005] This invention provides a data analysis method, system, electronic device, and storage medium to solve the aforementioned technical problems of difficulty in accurately understanding semantics, inefficient query generation, and difficulty in integration and expansion in data analysis.

[0006] This invention provides a data analysis method, comprising: acquiring user-input natural language query data; performing intent recognition on the natural language query data to obtain a target query intent and a query vector corresponding to the target query intent; retrieving multiple candidate dataset information from multiple preset dataset information based on the similarity between the query vector and multiple preset dataset vectors, each preset dataset information including metadata for describing the corresponding dataset in a database, the dataset vector being a vector format cached for the corresponding preset dataset information; scoring the matching degree of each candidate dataset information according to the target query intent to obtain selected dataset information; generating a structured semantic query language based on the target query intent and the selected dataset information, and converting the semantic query language into a physical query language of the corresponding database according to a preset mapping relationship, so as to query the corresponding dataset from the database to obtain the data to be analyzed; wherein, the preset mapping relationship is used to represent the mapping relationship between the semantic logic layer and the business physical layer.

[0007] In one embodiment of the present invention, the matching degree score of each candidate dataset information is calculated according to the target query intent to obtain selected dataset information, including: constructing prompt words according to the target query intent and each candidate dataset information, wherein the prompt words include dynamic weights corresponding to each type of metadata; inputting the prompt words into a large language model to obtain the matching degree score of each candidate dataset information; and determining the corresponding candidate dataset information as the selected dataset information according to each matching degree score.

[0008] In one embodiment of the present invention, multiple candidate dataset information is recalled from multiple preset dataset information based on the similarity between the query vector and multiple preset dataset vectors, including: determining the cosine similarity between the query vector and each of the dataset vectors respectively; and recalling multiple candidate dataset information from multiple preset dataset information according to the order of the cosine similarities.

[0009] In one embodiment of the present invention, a structured semantic query language is generated based on the target query intent and the selected dataset information, and the semantic query language is converted into a physical query language of the corresponding database according to a preset mapping relationship. This includes: generating a structured semantic query language based on preset business semantics, the target query intent, and the selected dataset information; converting the semantic query language into a physical query language of the corresponding database according to a preset mapping relationship; and optimizing the physical query language, wherein the optimization includes at least one of predicate pushdown, index usage, and parallel execution.

[0010] In one embodiment of the present invention, the intent recognition of the data query natural language to obtain the target query intent and the query vector corresponding to the target query intent includes: removing redundant information from the data query natural language to obtain an initial query statement; extracting the core query intent from the initial query statement; performing normalization transformation on the initial query statement according to the core query intent to obtain the target query intent; and converting the target query intent into a vector format to obtain the query vector.

[0011] In one embodiment of the present invention, after obtaining the data to be analyzed, the method further includes: displaying the data to be analyzed according to the data type characteristics of the data to be analyzed and / or the corresponding user memory preferences; If the user's data processing operation is received, the data to be analyzed is processed. The data processing operation includes at least one of the following: a drill-down operation to determine detailed data, a roll-up operation to determine summary data, and a slicing operation for multi-dimensional filtering.

[0012] This invention provides a data analysis system, comprising: a user interaction module for acquiring user-input natural language query data; an intent recognition module for performing intent recognition on the natural language query data to obtain a target query intent and a query vector corresponding to the target query intent; a recall module for recalling multiple candidate dataset information from multiple preset dataset information based on the similarity between the query vector and multiple preset dataset vectors, each preset dataset information including metadata describing the corresponding dataset in the database, and the dataset vector being a vector format cached for the corresponding preset dataset information; a reordering module for performing matching degree scoring on each candidate dataset information based on the target query intent to obtain selected dataset information; and a query construction module for generating a structured semantic query language based on the target query intent and the selected dataset information, and converting the semantic query language into a physical query language corresponding to the database according to a preset mapping relationship, so as to query the corresponding dataset from the database to obtain the data to be analyzed; wherein, the preset mapping relationship is used to represent the mapping relationship between the semantic logic layer and the business physical layer.

[0013] In one embodiment of the present invention, the system further includes: a semantic management module, used to manage the preset mapping relationship, the metadata and the preset business semantics, wherein the business semantics are used to generate the preset mapping relationship and the semantic query language; and a data visualization module, used to connect to multiple databases, execute the physical query language and preprocess the queried data to be analyzed, and display it.

[0014] The present invention provides an electronic device comprising: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the electronic device performs the data analysis method as described in any of the above embodiments.

[0015] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a computer processor, causes the computer to perform the data analysis method described in any of the above embodiments.

[0016] The beneficial effects of this invention are as follows: The data analysis method, system, electronic device, and storage medium proposed in this invention achieve the function of accurately understanding semantics and accurately locating the corresponding dataset through three-stage semantic matching, including intent recognition, recall and rearrangement of candidate dataset information; by generating a structured semantic query language and then converting it into the physical query language of the corresponding database, the dialect differences between different databases are resolved, and the query generation efficiency is improved; and the access of new data sources can be supported through the expansion of preset mapping relationships and preset dataset vectors. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0018] In the attached diagram: Figure 1 A schematic diagram of an exemplary system architecture provided in an embodiment of the present invention; Figure 2 This is a flowchart illustrating a data analysis method provided in one embodiment of the present invention; Figure 3 This is a schematic diagram illustrating the implementation process of the data analysis method provided in one embodiment of the present invention; Figure 4 This is a block diagram of a data analysis system provided in one embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of a computer system for an electronic device provided in one embodiment of the present invention. Detailed Implementation

[0019] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0020] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. The drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0021] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the invention.

[0022] Please see Figure 1 , Figure 1 This is a schematic diagram of an exemplary system architecture provided in an embodiment of the present invention. Figure 1 As shown, the system architecture may include a database 110 and a computer device 120. The computer device may be a general-purpose computer; the number of databases may be multiple.

[0023] For example, computer device 120 acquires natural language data query input from the user; performs intent recognition on the natural language data query to obtain the target query intent and the query vector corresponding to the target query intent; based on the similarity between the query vector and multiple preset dataset vectors, it recalls multiple candidate dataset information from multiple preset dataset information, each preset dataset information including metadata for describing the corresponding dataset in the database, and the dataset vector is a vector format cached for the corresponding preset dataset information; it scores the matching degree of each candidate dataset information according to the target query intent to obtain selected dataset information; it generates a structured semantic query language based on the target query intent and the selected dataset information, and converts the semantic query language into the physical query language of the corresponding database according to a preset mapping relationship, so as to query the corresponding dataset from the database and obtain the data to be analyzed; wherein, the preset mapping relationship is used to represent the mapping relationship between the semantic logic layer and the business physical layer.

[0024] In related technologies, data analysis suffers from technical problems such as difficulty in accurately understanding semantics, inefficient query generation, and difficulty in integration and expansion.

[0025] To address the aforementioned technical problems, this invention provides a data analysis method, system, electronic device, and storage medium. The implementation details of the technical solutions of the embodiments of this invention are described in detail below.

[0026] Please see Figure 2 , Figure 2 This is a flowchart illustrating a data analysis method provided in one embodiment of the present invention. Figure 2 As shown, in an exemplary embodiment, the data analysis method includes at least steps S210 to S250, which are described in detail below: Step S210: Obtain the user-input data query in natural language.

[0027] In one embodiment of the present invention, the natural language for data query includes data query text and data query voice.

[0028] In one embodiment of the present invention, a user interface is provided through a user interaction module, which enables users to input data and query natural language, thereby reducing the barrier to entry for users.

[0029] Step S220: Perform intent recognition on the natural language of the data query to obtain the target query intent and the query vector corresponding to the target query intent.

[0030] In one embodiment of the present invention, the intent recognition of the natural language of data query to obtain the target query intent and the query vector corresponding to the target query intent includes: removing redundant information from the natural language of data query to obtain an initial query statement; extracting the core query intent from the initial query statement; performing normalization transformation on the initial query statement according to the core query intent to obtain the target query intent; and converting the target query intent into a vector format to obtain the query vector.

[0031] In one embodiment of the present invention, the intent of the natural language query for data is identified by a large language model and contextual information, thereby obtaining a refined and standardized target query intent.

[0032] In one embodiment of the invention, the contextual information includes the user's query history and the current session context, thereby providing accuracy in semantic understanding.

[0033] In one embodiment of the present invention, the user's query history is recorded through a user interaction module, thereby providing query recommendations and quick repeat query functions.

[0034] Step S230: Based on the similarity between the query vector and multiple preset dataset vectors, retrieve multiple candidate dataset information from the multiple preset dataset information.

[0035] Each preset dataset information includes metadata describing the corresponding dataset in the database, and the dataset vector is a vector format of the corresponding preset dataset information cache.

[0036] In one embodiment of the present invention, multiple candidate dataset information is retrieved from multiple preset dataset information based on the similarity between the query vector and multiple preset dataset vectors, including: determining the cosine similarity between the query vector and each dataset vector respectively; and retrieving multiple candidate dataset information from multiple preset dataset information according to the order of the cosine similarities.

[0037] In one embodiment of the present invention, the information of the Top-K candidate datasets is quickly retrieved based on the ranking of each cosine similarity, that is, the information of a preset number of K candidate datasets with high cosine similarity, thereby improving the efficiency of subsequent processing.

[0038] In one embodiment of the present invention, metadata includes the name, fields, relationships, etc. of the dataset, providing a foundation for the semantic parsing process of generating a structured semantic query language.

[0039] In one embodiment of the present invention, multiple candidate dataset information are quickly recalled through the Embedding recall module.

[0040] Step S240: Based on the target query intent, score the matching degree of each candidate dataset information to obtain the selected dataset information.

[0041] In one embodiment of the present invention, matching degree scores are performed on each candidate dataset information according to the target query intent to obtain selected dataset information, including: constructing prompt words according to the target query intent and each candidate dataset information, wherein the prompt words include dynamic weights corresponding to each type of metadata; inputting the prompt words into a large language model to obtain matching degree scores for each candidate dataset information; and determining the corresponding candidate dataset information as the selected dataset information according to each matching degree score.

[0042] In one embodiment of the present invention, the candidate dataset information that best matches the target query intent is selected by comprehensively considering information such as the name, description, and fields contained in the candidate dataset information through a large language model.

[0043] Step S250: Generate a structured semantic query language based on the target query intent and the selected dataset information, and convert the semantic query language into the physical query language of the corresponding database according to the preset mapping relationship, so as to query the corresponding dataset from the database and obtain the data to be analyzed.

[0044] Among them, the preset mapping relationship is used to represent the mapping relationship between the semantic logic layer and the business physical layer.

[0045] In one embodiment of the present invention, the preset mapping relationship is obtained based on the schema mapping of the dataset.

[0046] In one embodiment of the present invention, a structured semantic query language is generated based on the target query intent and selected dataset information, and the semantic query language is converted into the physical query language of the corresponding database according to a preset mapping relationship. This includes: generating a structured semantic query language based on preset business semantics, target query intent and selected dataset information; converting the semantic query language into the physical query language of the corresponding database according to a preset mapping relationship; and optimizing the physical query language. The optimization includes at least one of predicate pushdown, index usage and parallel execution.

[0047] In one embodiment of the present invention, a semantic model is used to predefine the business semantics of an enterprise, including but not limited to business themes, indicator systems, dimensional structures, etc., thereby providing a unified semantic view.

[0048] In one embodiment of the present invention, the semantic model can be updated according to the business status, thereby ensuring the traceability and compatibility of semantic model changes.

[0049] In one embodiment of the present invention, the target query intent and selected dataset information are converted into a structured semantic query language through preset business semantics, that is, a semantic structured query language (SQL) that conforms to the semantic model is generated.

[0050] In one embodiment of the present invention, the semantic logic layer includes a logical table and logical fields corresponding to the dataset; the business physical layer includes a physical table and physical fields corresponding to the dataset.

[0051] In one embodiment of the present invention, semantic SQL is converted into the physical query language of the corresponding database, i.e., physical SQL, thereby resolving dialect differences between different databases.

[0052] In one embodiment of the present invention, predicate pushdown is used to characterize the initial execution of pushing down filtering conditions to the query phase, thereby reducing the amount of data processed subsequently.

[0053] In one embodiment of the invention, the index is used to select a suitable index to speed up the query.

[0054] In one embodiment of the present invention, parallel execution is used to decompose the query task into multiple subtasks, thereby processing them in parallel and improving query speed.

[0055] In one embodiment of the present invention, the database type includes, but is not limited to, relational database, data warehouse, and data lake.

[0056] In one embodiment of the present invention, the data query process is monitored and statistics are collected, thereby providing a basis for optimizing physical SQL.

[0057] In one embodiment of the present invention, physical SQL queries are executed efficiently through a query execution engine, and large amounts of data processing and streaming returns are supported.

[0058] In one embodiment of the present invention, after obtaining the data to be analyzed, the process includes: displaying the data to be analyzed according to the data type characteristics of the data to be analyzed and / or the corresponding user memory preferences; if a user's data processing operation is received, the data to be analyzed is processed, and the data processing operation includes at least one of the following: a drill-down operation for determining detailed data, a roll-up operation for determining summary data, and a slicing operation for multi-dimensional filtering.

[0059] In one embodiment of the present invention, after obtaining the data to be analyzed from the corresponding database, the data to be analyzed is preprocessed, including but not limited to data cleaning, format conversion, and aggregation calculation.

[0060] In one embodiment of the present invention, the invention can support the access of new data sources by configuring new preset mapping relationships. Semantic matching accuracy can be improved by updating preset dataset vectors; system capabilities can be upgraded by replacing large language models; and new visualization components can be added via plugins.

[0061] In one embodiment of the present invention, please refer to Figure 3 , Figure 3 This is a schematic diagram illustrating the implementation flow of a data analysis method provided in one embodiment of the present invention. In this embodiment, the enterprise possesses 30 datasets, including multiple subject areas such as sales, inventory, and customers. Figure 3As shown, in step S310, the user asks a question: The user enters the query "comparison of sales revenue in various regions last month" through the Web (webpage) interface. The user interaction module receives the input data query natural language and displays relevant query history for reference. In step S320, the large language model refines and optimizes the query: In the intent recognition module, the large language model refines the data query natural language to identify the core query intent: the time dimension is "last month", the geographical dimension is "various regions", the indicator is "sales revenue", and the analysis type is "comparison", thus obtaining the target query intent. In step S33... 0. Quickly retrieve information from multiple candidate datasets: Vectorize the target query intent and calculate the similarity with the dataset vectors corresponding to 30 datasets to retrieve the 5 most relevant candidate dataset information, including information corresponding to "Regional Sales Summary Table" and "Monthly Sales Details Table"; Step S340, Large Language Model Re-ranking to select the best dataset information: In the re-ranking module, the large language model scores the matching degree of the 5 candidate dataset information, identifies that the information corresponding to "Regional Sales Summary Table" contains the required region, month, and sales amount fields, gives it the highest score, and selects it to obtain the selected dataset information; Step S350: Semantic parsing and physical query construction: In the query construction module, semantic SQL conforming to the semantic model is constructed based on the target query intent and the selected dataset information, and then converted into physical SQL: "SELECT region, SUM(amount) FROM t_sales_region WHERE month='2024-11' GROUP BY region". The query optimizer suggests using the month index to improve query performance. The physical SQL means selecting the sum of the region and amount fields, with the data source being t_sales_region (regional sales summary table), and the filtering condition being data from November 2024, grouped and returned by region. Step S360: Query result visualization: The query is executed through the data visualization module, and the results are obtained. A bar chart is automatically selected to display the sales comparison of each region, and a drill-down interactive function to the city level is provided. The entire query process takes less than 1 second, providing a good user experience.

[0062] Please see Figure 4 , Figure 4 This is a block diagram of a data analysis system provided in one embodiment of the present invention. This system can be applied to... Figure 1 The implementation environment shown is specifically configured in computer device 120. This system can also be applied to other exemplary implementation environments and specifically configured in other devices; this embodiment does not limit the implementation environment to which the system is applicable.

[0063] like Figure 4 As shown, a data analysis system 400 according to an embodiment of the present invention includes: a user interaction module 410, an intent recognition module 420, a recall module 430, a rearrangement module 440, and a query construction module 450.

[0064] Among them, the user interaction module 410 is used to obtain user input data query natural language; The intent recognition module 420 is used to perform intent recognition on the natural language of data query to obtain the target query intent and the query vector corresponding to the target query intent. The recall module 430 is used to recall multiple candidate dataset information from multiple preset dataset information based on the similarity between the query vector and multiple preset dataset vectors. Each preset dataset information includes metadata describing the corresponding dataset in the database, and the dataset vector is a vector format cached for the corresponding preset dataset information. The rearrangement module 440 is used to score the matching degree of each candidate dataset information according to the target query intent, and obtain the selected dataset information. The query construction module 450 is used to generate a structured semantic query language based on the target query intent and selected dataset information, and to convert the semantic query language into the physical query language of the corresponding database according to the preset mapping relationship, so as to query the corresponding dataset from the database and obtain the data to be analyzed. Among them, the preset mapping relationship is used to represent the mapping relationship between the semantic logic layer and the business physical layer.

[0065] It should be noted that the data analysis system and the data analysis method provided in the above embodiments belong to the same concept. The specific ways in which each module and unit performs operations have been described in detail in the method embodiments and will not be repeated here. In practical applications, the data analysis system provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the system can be divided into different functional modules to complete all or part of the functions described above. This is not a limitation here.

[0066] Embodiments of the present invention also provide an electronic device, including: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the electronic device enables the data analysis methods provided in the above embodiments.

[0067] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of a computer system for an electronic device provided in one embodiment of the present invention. Figure 5The computer system 500 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.

[0068] like Figure 5 As shown, the computer system 500 includes a central processing unit 501, which can perform various appropriate actions and processes based on a program stored in the read-only memory 502 or a program loaded from the storage section 508 into the random access memory 503, such as performing the methods described in the above embodiments. The random access memory 503 also stores various programs and data required for system operation. The central processing unit 501, the read-only memory 502, and the random access memory 503 are interconnected via a bus 504. An input / output interface 505 is also connected to the bus 504.

[0069] The following components are connected to the input / output interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the input / output interface 505 as needed. A removable medium 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 510 as needed so that computer programs read from it can be installed into the storage section 508 as needed.

[0070] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing computer programs for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable medium 511. When the computer program is executed by central processing unit 501, it performs various functions defined in the system of the present invention.

[0071] The computer-readable medium shown in the embodiments of the present invention can be a computer-readable signal medium, a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory (EPROM), flash memory, optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. Computer programs contained on computer-readable media can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0072] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. Each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0073] The units described in the embodiments of the present invention can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself. Therefore, the technical solutions according to the embodiments of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, portable hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the method according to the embodiments of the present invention.

[0074] Another aspect of the present invention provides a computer-readable storage medium storing a computer program thereon, which, when executed by a computer's processor, causes the computer to perform the data analysis methods provided in the various embodiments described above. This computer-readable storage medium may be included in the electronic devices described in the above embodiments, or it may exist independently and not incorporated into the electronic device.

[0075] In the above embodiments, unless otherwise specified, the use of ordinal numbers such as "first" and "second" to describe common objects only indicates that they refer to different instances of the same object, rather than indicating that the objects being described must be in a given order, whether temporally, spatially, sequentially, or in any other way.

[0076] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A data analysis method, characterized by, The method comprises: acquiring a user-input data query natural language; performing intent recognition on the data query natural language to obtain a target query intent and a query vector corresponding to the target query intent; recalling a plurality of candidate dataset information from a plurality of preset dataset information according to a similarity between the query vector and a plurality of preset dataset vectors, each of the preset dataset information comprising metadata for describing a corresponding dataset in a database, and the dataset vector being a vector format cached for the corresponding preset dataset information; performing matching degree scoring on each of the candidate dataset information according to the target query intent to obtain selected dataset information; generating a structured semantic query language based on the target query intent and the selected dataset information, and converting the semantic query language into a physical query language of a corresponding database according to a preset mapping relationship to query the corresponding dataset from the database to obtain to-be-analyzed data; wherein the preset mapping relationship represents a mapping relationship between a semantic logic layer and a business physical layer.

2. The data analysis method of claim 1, wherein, The matching degree scoring on each of the candidate dataset information according to the target query intent to obtain selected dataset information comprises: constructing a prompt word according to the target query intent and each of the candidate dataset information, the prompt word comprising a dynamic weight corresponding to each type of metadata; inputting the prompt word into a large language model to obtain a matching degree score of each of the candidate dataset information; determining the corresponding candidate dataset information as the selected dataset information according to each of the matching degree scores.

3. The data analysis method of claim 2, wherein, The recalling of a plurality of candidate dataset information from a plurality of preset dataset information according to a similarity between the query vector and a plurality of preset dataset vectors comprises: respectively determining a cosine similarity between the query vector and each of the dataset vectors; recalling a plurality of candidate dataset information from a plurality of preset dataset information according to a ranking of each of the cosine similarities.

4. The data analysis method of claim 1, wherein, The generating of a structured semantic query language based on the target query intent and the selected dataset information, and the converting of the semantic query language into a physical query language of a corresponding database according to a preset mapping relationship comprises: generating a structured semantic query language based on a preset business semantic, the target query intent and the selected dataset information; converting the semantic query language into a physical query language of a corresponding database according to a preset mapping relationship, and optimizing the physical query language, the optimization comprising at least one of predicate pushdown, index usage and parallel execution.

5. The data analysis method of claim 1, wherein, The performing of intent recognition on the data query natural language to obtain a target query intent and a query vector corresponding to the target query intent comprises: removing redundant information in the data query natural language to obtain an initial query sentence; extracting a core query intent in the initial query sentence; performing normalized conversion on the initial query sentence according to the core query intent to obtain a target query intent; converting the target query intent into a vector format to obtain a query vector.

6. The data analysis method according to any one of claims 1 to 5, characterized in that, After obtaining the to-be-analyzed data, further comprising: displaying the to-be-analyzed data according to a data type feature and / or a corresponding user memory preference of the to-be-analyzed data; If the data processing operation of the user is received, the data to be analyzed is processed, the data processing operation including at least one of a drill-down operation for determining detailed data, a roll-up operation for determining summary data, and a slicing operation for multi-dimensional filtering.

7. A data analysis system, characterized by The system comprises: a user interaction module configured to obtain a data query natural language input by a user; an intent recognition module configured to perform intent recognition on the data query natural language to obtain a target query intent and a query vector corresponding to the target query intent; a recall module configured to recall a plurality of candidate data set information from a plurality of preset data set information according to a similarity between the query vector and a plurality of preset data set vectors, each of the preset data set information including metadata for describing a corresponding data set in a database, and the data set vector being a vector format cached for the corresponding preset data set information; a rearrangement module configured to perform a matching degree score on each of the candidate data set information according to the target query intent to obtain selected data set information; a query construction module configured to generate a structured semantic query language based on the target query intent and the selected data set information, and convert the semantic query language into a physical query language of a corresponding database according to a preset mapping relationship to query the corresponding data set from the database to obtain data to be analyzed; wherein the preset mapping relationship represents a mapping relationship between a semantic logic layer and a business physical layer.

8. The data analysis system of claim 7, wherein, The system further comprises: a semantic management module configured to manage the preset mapping relationship, the metadata, and a preset business semantic, the business semantic being used to generate the preset mapping relationship and the semantic query language; a data visualization module configured to connect a plurality of databases, execute the physical query language, and pre-process the queried data to be analyzed, and display the data to be analyzed.

9. An electronic device, comprising: The electronic device comprises: one or more processors; a storage device configured to store one or more programs, which, when executed by the one or more processors, cause the electronic device to implement the data analysis method according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that, A computer program is stored thereon, which, when executed by a processor of a computer, causes the computer to perform the data analysis method according to any one of claims 1 to 6.

Images