Visual data analysis method and device based on associated query

Abstract

The invention discloses a visual data analysis method and device based on associated query, and is applied to the technical field of data processing. The method comprises the following steps: displaying a plurality of first data sources in a lightweight database on an interactive interface, wherein the plurality of first data sources are obtained by performing data conversion on a plurality of second data sources; receiving a first input of a plurality of target data tables selected by a user from the plurality of first data sources, a plurality of target fields in each target data table and a target association mode; in response to the first input, matching a plurality of target fields selected by the user to obtain a matching relationship among the plurality of target fields; generating a first association query statement according to the plurality of target data tables, the matching relationship and the target association mode; and executing the first associated query statement through the lightweight database to obtain and display a first associated query result. According to the embodiment of the invention, the technical requirements on data analysts can be reduced, and the data analysis efficiency can be improved.

Description

Technical Field

[0001] The present application belongs to the field of data processing technology, and in particular, relates to a visual data analysis method and device based on association query. Background Art

[0002] With the development of digital information technology, data processing and analysis are becoming increasingly important and play a huge role in all walks of life.

[0003] At present, there are many tools for data analysis in a single-machine environment. However, existing data analysis tools often require complex installation and configuration processes, and have high technical requirements for data analysts. In addition, existing data analysis tools may not support related query and analysis of multiple data sources. Even if some data analysis tools support related query and analysis of multiple data sources, users are usually required to write complex query statements, resulting in low efficiency of data analysis. Summary of the invention

[0004] The embodiments of the present application provide a method, apparatus, device, computer-readable storage medium, and computer program product for visual data analysis based on associative query, which can reduce the technical requirements for data analysts and improve data analysis efficiency.

[0005] In a first aspect, an embodiment of the present application provides a visual data analysis method based on association query, the method comprising:

[0006] Displaying multiple first data sources in a lightweight database on an interactive interface, wherein the multiple first data sources are obtained by converting multiple second data sources according to a data format and structure corresponding to the lightweight database, and the multiple second data sources are heterogeneous data sources;

[0007] receiving a first input from a user selecting a plurality of target data tables, a plurality of target fields in each of the target data tables, and a target association method from the plurality of first data sources;

[0008] In response to the first input, matching the multiple target fields selected by the user to obtain a matching relationship between the multiple target fields;

[0009] Generate a first association query statement according to the multiple target data tables, the matching relationship and the target association mode;

[0010] The first associated query statement is executed through the lightweight database to obtain and display a first associated query result.

[0011] In a possible implementation, in response to the first input, matching the multiple target fields selected by the user to obtain a matching relationship between the multiple target fields includes:

[0012] In response to the first input, for each first target field in a first target data table, determining a fuzzy matching degree between the first target field and a plurality of target fields in each second target data table, wherein the first target data table is any one of the plurality of target data tables, the first target field is any one of the plurality of target fields in the first target data table, and the second target data table is any one of the plurality of target data tables except the first target data table;

[0013] For each of the first target fields in the first target data table and each of the second target data table, the target field corresponding to the fuzzy matching degree with the largest value among the multiple fuzzy matching degrees is determined as the matching field of the first target field, so as to obtain the matching relationship between the multiple target fields.

[0014] In a possible implementation manner, after obtaining the matching relationship between the plurality of target fields, the method further includes:

[0015] Displaying the matching relationship on the interactive interface;

[0016] receiving a second input from a user for adjusting the matching relationship;

[0017] In response to the second input, updating the matching relationship;

[0018] The generating a first association query statement according to the multiple target data tables, the matching relationship and the target association mode includes:

[0019] A first association query statement is generated according to the multiple target data tables, the updated matching relationship and the target association mode.

[0020] In a possible implementation, the interactive interface includes a data table selection area, a field selection area, and an association method selection area, the first input includes a first sub-input, a second sub-input, and a third sub-input; the receiving of the first input of the user selecting multiple target data tables, multiple target fields in each of the target data tables, and target association methods from the multiple first data sources includes:

[0021] Receive a first sub-input of a user dragging multiple target data tables from the multiple first data sources to the data table selection area, a second sub-input of a user dragging multiple target fields in each of the target data tables to the field selection area, and a third sub-input of a user selecting a target association method in the association method selection area.

[0022] In a possible implementation, before displaying the plurality of first data sources in the lightweight database on the interactive interface, the method further includes:

[0023] displaying the interactive interface;

[0024] receiving a third input from a user in the interactive interface for importing the plurality of second data sources;

[0025] In response to the third input, performing data conversion on the plurality of second data sources according to a data format and structure corresponding to the lightweight database to obtain the plurality of first data sources;

[0026] The plurality of first data sources are imported into the lightweight database.

[0027] In a possible implementation, in response to the third input, converting the plurality of second data sources according to a data format and structure corresponding to the lightweight database to obtain the plurality of first data sources includes:

[0028] In response to the third input, respectively verify the data in the plurality of second data sources to obtain a data verification result;

[0029] When the data verification result is that the verification is passed, data conversion is performed on the multiple second data sources according to the data format and structure corresponding to the lightweight database to obtain the multiple first data sources.

[0030] In a possible implementation, the interactive interface includes a query statement editing area, and after the interactive interface displays the plurality of first data sources in the lightweight database, the method further includes:

[0031] receiving a fourth input from a user to edit a second associated query statement in the query statement editing area;

[0032] In response to the fourth input, determining the second associated query statement;

[0033] The second associated query statement is executed through the lightweight database to obtain and display a second associated query result.

[0034] In a second aspect, an embodiment of the present application provides a visual data analysis device based on association query, the device comprising:

[0035] A first display module is used to display multiple first data sources in the lightweight database on an interactive interface, wherein the multiple first data sources are obtained by converting multiple second data sources according to a data format and structure corresponding to the lightweight database, and the multiple second data sources are heterogeneous data sources;

[0036] A first receiving module is used to receive a first input from a user selecting a plurality of target data tables, a plurality of target fields in each of the target data tables, and a target association method from the plurality of first data sources;

[0037] a matching module, configured to match the plurality of target fields selected by the user in response to the first input, and obtain a matching relationship between the plurality of target fields;

[0038] A generating module, configured to generate a first associated query statement according to the plurality of target data tables, the matching relationship and the target association mode;

[0039] The first execution module is used to execute the first associated query statement through the lightweight database to obtain and display a first associated query result.

[0040] In a third aspect, an embodiment of the present application provides an electronic device, the device comprising: a processor and a memory storing computer program instructions;

[0041] When the processor executes the computer program instructions, the processor implements any possible implementation method of the first aspect described above.

[0042] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, implements a method in any possible implementation method of the first aspect described above.

[0043] In a fifth aspect, an embodiment of the present application provides a computer program product. When instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes a method as any possible implementation method in the first aspect above.

[0044] The visual data analysis method, device, equipment, computer-readable storage medium and computer program product based on associative query of the embodiments of the present application use a lightweight database for data analysis, without the need for complex installation and configuration processes, and with low technical requirements for data analysts. In the scenario of associative query and analysis of multiple data sources, by providing an interactive interface, the user does not need to write complex associative query statements. By selecting multiple target data tables, multiple target fields in each target data table and target association methods in the interactive interface, the first associative query result can be obtained and displayed, and the technical requirements for data analysts are low. In addition, by automatically generating the first associative query statement based on multiple target data tables, matching relationships and target association methods, the efficiency of data analysis can be improved compared to the user manually writing the associative query statement. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0046] Figure 1 It is a flowchart of a visual data analysis method based on association query provided by an embodiment of the present application;

[0047] Figure 2 It is a schematic diagram of a process of importing data into a lightweight database provided by an embodiment of the present application;

[0048] Figure 3 It is a flowchart of determining a matching relationship between multiple target fields provided in an embodiment of the present application;

[0049] Figure 4 It is a flowchart of another visual data analysis method based on association query provided in an embodiment of the present application;

[0050] Figure 5 It is a structural schematic diagram of a visual data analysis device based on association query provided in an embodiment of the present application;

[0051] Figure 6 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0053] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0054] In addition, the acquisition, storage, use, and processing of data in the technical solution of this application comply with the relevant provisions of national laws and regulations.

[0055] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0056] In stand-alone mode, there is a growing demand for lightweight data analysis services. However, existing data analysis tools have many problems when processing data in a stand-alone environment. Existing data analysis tools often require complex installation and configuration processes, and have high technical requirements for users. In addition, these tools may encounter performance bottlenecks when processing large-scale data, resulting in low analysis efficiency.

[0057] At the same time, existing data analysis tools also have certain functional limitations. They may not meet users' needs for advanced functions such as support for multiple data sources, intelligent association field recognition, fuzzy matching, and real-time data analysis. These problems require users to spend a lot of time and energy to process data when conducting data analysis, affecting work efficiency and the accuracy of data analysis.

[0058] Specifically, the prior art includes the following disadvantages:

[0059] Poor usability: Existing data analysis tools often require users to have a certain technical background, such as being familiar with SQL syntax or programming languages, before they can perform data analysis operations. This makes it difficult for non-technical users to quickly get started with the tools.

[0060] Single function: Existing data analysis tools have relatively single functions. For example, they can only support specific types of data sources or association methods, and cannot meet users' diverse data analysis needs.

[0061] Inefficiency: Existing data analysis tools may encounter performance bottlenecks when processing large-scale data, resulting in low analysis efficiency and affecting user work efficiency;

[0062] Lack of visualization: Existing data analysis tools often only provide data tables or text results, lack of visual display, and it is difficult to intuitively understand the data and analysis results.

[0063] Therefore, a solution specifically for lightweight data analysis services in stand-alone mode is needed, which can provide a simple and easy-to-use interface, efficient data processing capabilities and rich functions to meet users' needs for data analysis in a stand-alone environment.

[0064] Thus, in order to solve the problems of the prior art, the embodiments of the present application provide a visual data analysis method, device, equipment, computer-readable storage medium and computer program product based on associated query. Among them, the visual data analysis method based on associated query can be applied to audit scenarios. In addition, the visual data analysis method based on associated query can be executed by a data analysis associated query system. The data analysis associated query system can provide a data integration wizard, support users to select data sources and tables, and set association relationships, so as to facilitate users to perform data integration processing. Among them, the data analysis associated query system can include a front-end interactive interface and a background processing system. The background processing system can include a data processing subsystem and a lightweight database. Specifically, the interactive interface can be used to provide a visual interface to the user and perform human-computer interaction with the user. The interactive interface can adopt a modern graphical user interface design, provide a concise and clear operation process and a rich visual chart display, support drag and drop, click and other operation modes, facilitate users to quickly build data analysis and associated query tasks, and enable users to easily perform data analysis and associated query operations. In addition, the data processing subsystem can be used to implement the background functions corresponding to the interactive interface. The lightweight database can be used to execute associated query statements.

[0065] In addition, the data analysis and association query system in the embodiment of the present application can support multiple operating systems such as Windows, Linux, Mac OS, etc., which is convenient for users to perform data analysis and association query on different platforms. The data analysis and association query system can also provide cross-platform client and server applications to facilitate users to perform remote access and management.

[0066] In addition, the data analysis and association query system in the embodiment of the present application adopts strict security measures to protect the security and privacy of user data. By encrypting the storage and transmission of user data, the security and privacy of data can be ensured. By providing user identity authentication and authorization functions, it can be ensured that only authorized users can access and operate data.

[0067] The following is an introduction to the visual data analysis method based on association query provided in the embodiment of the present application. Figure 1 A flow chart of a visual data analysis method based on association query provided in an embodiment of the present application is shown.

[0068] like Figure 1 As shown, the visual data analysis method based on association query provided by the embodiment of the present application includes the following steps:

[0069] S110, displaying multiple first data sources in the lightweight database on an interactive interface, where the multiple first data sources are obtained by converting multiple second data sources according to a data format and structure corresponding to the lightweight database, and the multiple second data sources are heterogeneous data sources;

[0070] S120, receiving a first input from a user selecting a plurality of target data tables, a plurality of target fields in each target data table, and a target association method from a plurality of first data sources;

[0071] S130, in response to the first input, matching multiple target fields selected by the user to obtain a matching relationship between the multiple target fields;

[0072] S140, generating a first association query statement according to multiple target data tables, matching relationships, and target association modes;

[0073] S150: Execute the first associated query statement through the lightweight database to obtain and display the first associated query result.

[0074] The visual data analysis method based on associative query in the embodiment of the present application uses a lightweight database for data analysis, does not require a complicated installation and configuration process, and has low technical requirements for data analysts. In the scenario of associative query and analysis of multiple data sources, by providing an interactive interface, the user does not need to write complex associative query statements. By selecting multiple target data tables, multiple target fields in each target data table, and target association methods in the interactive interface, the first associative query result can be obtained and displayed, which has low technical requirements for data analysts. In addition, by automatically generating the first associative query statement based on multiple target data tables, matching relationships, and target association methods, the efficiency of data analysis can be improved compared to the user manually writing the associative query statement.

[0075] The specific implementation methods of the above steps are introduced below.

[0076] In some embodiments, in S110, a lightweight database refers to a database system that is relatively lightweight in terms of resource occupation, deployment, and management, and is generally characterized by being small in size, easy to expand, and simple to use. A lightweight database may be, for example, a SQLite database. In addition, the data source may be a source of raw data used in data analysis. Multiple second data sources may include relational databases, non-relational databases, Excel files, CSV files, and the like. For each second data source, a first data source corresponding to the second data source may be obtained by performing data conversion on the second data source according to the data format and structure corresponding to the lightweight database. The first data source may be stored in a lightweight database.

[0077] As an example, the interactive interface may include a data source display area for displaying a plurality of first data sources in the lightweight database.

[0078] In addition, the data analysis association query system may further include a data import wizard, which may support automatic detection of data source formats and data types, as well as unified management and integration of different data sources. Based on this, in order to simplify the data import process and facilitate users to perform cross-data source data association query and analysis, in some embodiments, before the above S110, a data import process may also be included. Figure 2 As shown, the data import process may include the following steps:

[0079] S21, displaying an interactive interface;

[0080] S22, receiving a third input from the user for importing a plurality of second data sources in the interactive interface;

[0081] S23, in response to the third input, performing data conversion on the plurality of second data sources according to the data format and structure corresponding to the lightweight database to obtain a plurality of first data sources;

[0082] S24. Import multiple first data sources into the lightweight database.

[0083] Here, the interactive interface may include an "import" control. The third input includes, but is not limited to: the user's touch input to the "import" control through a touch device such as a finger or a stylus, a voice command input by the user, a specific gesture input by the user, other feasible inputs, etc. The third input can be determined according to actual usage needs, and the embodiment of the present application is not limited. The specific gesture in the embodiment of the present application can be any one of a single-click gesture, a sliding gesture, a drag gesture, a pressure recognition gesture, a long press gesture, an area change gesture, a double-press gesture, and a double-click gesture; the click input in the embodiment of the present application can be a single-click input, a double-click input, or any number of click inputs, etc., and can also be a long press input or a short press input. Based on this, the above-mentioned third input can, for example, be: a user's click input to the "import" control.

[0084] As an example, when the interactive interface is displayed, the user can click the "Import" control. After the user clicks the "Import" control, multiple data sources in the server can be displayed. The user can select multiple second data sources from the multiple data sources, for example, select one or several tables of one database and one or several tables of another database, and click the "OK" control. Afterwards, the data import wizard can automatically detect the data source formats and data types corresponding to the multiple second data sources, and perform data conversion on the multiple second data sources according to the data format and structure corresponding to the lightweight database, to obtain multiple first data sources with the same data structure and format, and import the multiple first data sources with the same data structure and format into the lightweight database. After the multiple first data sources are imported into the lightweight database, the multiple first data sources can be displayed in the data source display area of ​​the interactive interface.

[0085] In this way, by responding to the third input, the data source formats and data types corresponding to multiple second data sources are automatically detected based on the data import wizard, and multiple different second data sources are uniformly managed and integrated to obtain multiple first data sources, and the multiple first data sources are imported into the lightweight database, which can simplify the data import process and facilitate users to perform data association queries and analysis across data sources.

[0086] Based on this, in order to further improve the efficiency of data analysis, in some embodiments, in response to the third input, the data of the multiple second data sources are converted according to the data format and structure corresponding to the lightweight database to obtain the multiple first data sources, which may specifically include:

[0087] In response to the third input, respectively verify the data in the plurality of second data sources to obtain data verification results;

[0088] When the data verification result is that the verification is passed, data conversion is performed on the multiple second data sources according to the data format and structure corresponding to the lightweight database to obtain multiple first data sources.

[0089] Here, data verification may include verifying whether the file format of the second data source is correct, whether the format of the data table is correct, whether the data format in the data table is correct, whether the data in the data table is complete, etc. The data verification result may include verification pass and verification fail.

[0090] In addition, the data import wizard may also support a data verification function for respectively verifying the data in the plurality of second data sources, and, if the verification passes, performing data conversion on the plurality of second data sources.

[0091] In this way, by first performing data verification and then performing data conversion when the data verification passes, the success of data conversion can be guaranteed and the efficiency of data analysis can be further improved.

[0092] In addition, the data analysis and association query system may further include an alarm module. When the data verification result is a verification failure, the alarm module may be used to warn the user so that the user can process the data in the second data source in a timely manner, thereby improving data analysis efficiency.

[0093] In some embodiments, in S120, the target data table may be a data table selected by the user for data association analysis; the target field may be an association field selected by the user for association analysis. Among them, the association field may be a field in the database table used to establish an association relationship, such as an identifier, name, date, etc. In addition, the target association method may be an association method selected by the user from multiple association methods. Among them, multiple association methods may include left association, right association, inner association, outer association, etc. In addition, multiple association methods may also include user-defined association methods to select the most appropriate association method according to specific data analysis requirements.

[0094] As an example, the data source display area may display detailed information corresponding to multiple first data sources. The detailed information may include information such as the name of the data table, the fields and data in the data table, etc. The user may select multiple target data tables, multiple target fields in each target data table, and target association methods based on the detailed information of the first data source.

[0095] Specifically, the interactive interface may include a data table selection area, a field selection area and an association method selection area, and the first input includes a first sub-input for selecting a target data table, a second sub-input for selecting a target field and a third sub-input for selecting a target association method.

[0096] Based on this, in some embodiments, the above S120 may specifically include:

[0097] Receive the user dragging multiple target data tables from multiple first data sources to the first sub-input of the data table selection area, dragging multiple target fields in each target data table to the second sub-input of the field selection area, and selecting the third sub-input of the target association method in the association method selection area.

[0098] Here, the first sub-input may be a drag input for the user to select a target data table. The second sub-input may be a drag input for the user to select a target field. The third sub-input may be a selection input for the user to select a target association method from multiple association methods.

[0099] For example, the specific method for a user to select multiple target data tables can be as follows: the user can first select a target data table in the data source display area, and then drag the target data table to the data table selection area, and display the data table name of the target data table in the data table selection area. After that, the user can repeat the above steps to drag multiple target data tables to the data table selection area. After the user clicks the "OK" control corresponding to the data table selection area, the data processing subsystem can determine the multiple target data tables.

[0100] Exemplarily, the specific method for a user to select multiple target fields in each target data table can be as follows: for each target data table, the user can first select a target field in the data source display area, then drag the target field to the field selection area, and display the field name of the target field in the field selection area. The field name can carry a data table label. After that, the user can repeat the above steps to drag the multiple target fields corresponding to each of the multiple target data tables to the field selection area. After the user clicks the "OK" control corresponding to the field selection area, the data processing subsystem can determine the multiple target fields corresponding to each of the multiple target data tables.

[0101] For example, the specific method for the user to select the target association method can be as follows: a plurality of association methods can be displayed in the association method selection area, and the user can click on one of the plurality of association methods to obtain the target association method. After the user clicks on the "OK" control corresponding to the association method selection area, the data processing subsystem can determine the target association method.

[0102] In some embodiments, in S130, the matching relationship between multiple target fields can represent the association relationship between multiple target data tables. For example, if the user selects three fields such as name, age, and gender in the first data table, and also selects three fields such as name, age, and gender in the second data table, the name in the first data table and the name in the second data table can have a first matching relationship; the age in the first data table and the age in the second data table can have a second matching relationship; the gender in the first data table and the gender in the second data table can have a third matching relationship. The first data table and the second data table can be associated through the first matching relationship, the second matching relationship, and the third matching relationship. In addition, if two target fields have a matching relationship, the two target fields can be associated fields.

[0103] As an example, after determining multiple target data tables, multiple target fields, and target association methods, the data processing subsystem may first match the multiple target fields and automatically identify the associated fields in the data source.

[0104] Based on this, in order to improve the accuracy and efficiency of the associated query, in some embodiments, Figure 3 As shown, the above S130 may specifically include the following steps:

[0105] S31. In response to the first input, for each first target field in the first target data table, determine the fuzzy matching degree between the first target field and multiple target fields in each second target data table, where the first target data table is any one of the multiple target data tables, the first target field is any one of the multiple target fields in the first target data table, and the second target data table is any one of the multiple target data tables except the first target data table;

[0106] S32. For each first target field in the first target data table and each second target data table, determine the target field corresponding to the fuzzy matching degree with the largest value among multiple fuzzy matching degrees as the matching field of the first target field, and obtain the matching relationship between the multiple target fields.

[0107] Here, the two target fields can be fuzzily matched by the fuzzy matching algorithm. Through fuzzy matching, even if the values ​​of the two fields are not exactly the same, the matching relationship between them can be identified.

[0108] Exemplarily, if the multiple target data tables include a first data table, a second data table, and a third data table, and the first data table is determined as the first target data table, the second target data table may include the second data table and the third data table. If the multiple target fields in the first data table include name 1, gender 1, and age 1, the multiple target fields in the second data table include name 2, gender 2, and age 2, and the multiple target fields in the third data table include name 3, gender 3, and age 3, then when the first target field is name 1, for the second data table, the fuzzy matching degree between name 1 and name 2, gender 2, and age 2 can be calculated respectively to obtain three first fuzzy matching degrees, and the field corresponding to the fuzzy matching degree with the largest value among the three first fuzzy matching degrees is determined as the matching field of name 1. In this example, the matching field of name 1 may be name 2. Similarly, for the third data table, the fuzzy matching degree between name 3, gender 3, and age 3 can be calculated respectively to obtain three second fuzzy matching degrees, and the field corresponding to the fuzzy matching degree with the largest value among the three second fuzzy matching degrees is determined as the matching field of name 1. In this example, the matching field of name 1 may be name 3. In this way, it can be determined through fuzzy matching that the matching fields of name 1 include name 2 and name 3.

[0109] By means of the above method, at least one matching field of each first target field in the first target data table can be determined, thereby obtaining a matching relationship between all target fields selected by the user.

[0110] In this way, the matching relationship between multiple target fields is automatically determined by the fuzzy matching algorithm, which can reduce the workload of users manually setting the associated fields and improve the efficiency of associated queries. In addition, by performing fuzzy matching on multiple target fields, even if the values ​​of two fields are not exactly the same, the matching relationship between them can be identified, which improves the flexibility and accuracy of associated queries.

[0111] In addition, in some embodiments, a machine learning algorithm can be used to automatically analyze field relationships in a data source and identify potential associated fields based on the user's query history and data analysis requirements.

[0112] Specifically, the associated fields can be annotated through the user's query history, and the annotated associated fields are used as training data to train the machine learning model to obtain a target machine learning model that can identify associated fields. By inputting multiple target fields into the target machine learning model, the matching relationship between the multiple target fields can be obtained.

[0113] Based on this, in order to further improve the accuracy of the associated query, in some embodiments, after the above S130, the following may also be included:

[0114] Display matching relationships in the interactive interface;

[0115] receiving a second input from a user for adjusting the matching relationship;

[0116] In response to the second input, the matching relationship is updated.

[0117] Here, the interactive interface may include a matching relationship display area. In addition to displaying the matching relationship, the matching relationship display area may also display the fuzzy matching degree corresponding to the matching relationship. The display form of the matching relationship may be in the form of a corresponding relationship or in the form of a query statement, which is not limited here. The display form of the corresponding relationship may be, for example, first data table. Name 1 - second data table. Name 2 - third data table. Name 3. In addition, the second input may be an editing input for the user to adjust the matching relationship in the matching relationship display area.

[0118] After the user adjusts the matching relationship, the updated matching relationship may be displayed.

[0119] In this way, by visually displaying the matching relationship, it is convenient for users to adjust the matching relationship and further improve the accuracy of the associated query.

[0120] In some embodiments, in S140, after determining multiple target data tables, matching relationships, and target association modes, the data processing subsystem can automatically generate a first association query statement according to the multiple target data tables, matching relationships, and target association modes, and send the first association query statement to the lightweight database. When the lightweight database is a SQLite database, the first association query statement can be a SQL statement.

[0121] In addition, as described above, the user can adjust the matching relationship. Therefore, if the matching relationship is updated, in some embodiments, the above S140 may specifically include:

[0122] A first association query statement is generated according to the multiple target data tables, the updated matching relationship and the target association mode.

[0123] In this way, by generating the first associated query statement based on the updated matching relationship, the accuracy of the associated query can be further improved.

[0124] In some embodiments, in S150, after generating the first associated query statement, the first associated query statement can be executed through the lightweight database to obtain the first associated query result. After obtaining the first associated query result, the first associated query result can be stored in the lightweight database for the next data query and analysis. In addition, the first associated query result can be displayed in the form of charts, etc., to make the data more intuitive and easy to understand. The embodiment of the present application supports a variety of data visualization chart types, including bar charts, line charts, pie charts, etc., to facilitate users to choose the most appropriate chart display method. The embodiment of the present application can also provide a customized function for data visualization charts, supporting users to adjust the chart style and color according to specific needs. In addition, the interactive interface may include an "export" control for users to share data and generate reports based on the first associated query results.

[0125] In addition, the embodiment of the present application also supports real-time data analysis, that is, analyzing and processing the data, and immediately obtaining the analysis results, such as real-time monitoring of data change trends. In addition, the embodiment of the present application also supports filtering and sorting the first associated query results, so that users can quickly find the required information. Specifically, the interactive interface can include a variety of data filtering conditions and sorting options, support users to flexibly filter and sort data according to specific needs, and update and display the filtering and sorting results in real time, so that users can quickly find the required information.

[0126] In addition, the interactive interface may also include a query statement editing area. Based on this, in order to improve the user experience of the data analysis association query system, as another implementation of the present application, the present application also provides another implementation of a visual data analysis method based on association query, see the following embodiments for details.

[0127] See also Figure 4 The visual data analysis method based on association query provided in the embodiment of the present application may include steps S410-S440.

[0128] S410, the interactive interface displays multiple first data sources in the lightweight database.

[0129] S420: Receive a fourth input from the user for editing the second associated query statement in the query statement editing area.

[0130] Here, the fourth input may be an editing input of the second associated query statement by the user in the query statement editing area by typing, wherein the second associated query statement may be the same as or different from the first associated query statement, which is not limited here.

[0131] S430: In response to the fourth input, determine a second associated query statement.

[0132] After the user clicks the "OK" control corresponding to the query statement editing area, the data processing subsystem may determine the second associated query statement and send the second associated query statement to the lightweight database.

[0133] S440: Execute the second associated query statement through the lightweight database to obtain and display the second associated query result.

[0134] In addition, the other steps of the method in the embodiment of the present application can be seen above Figure 1 The relevant description of the illustrated embodiment will not be repeated here.

[0135] The embodiment of the present application provides a query statement editing area for the user, allowing the user to write the second associated query statement by himself, providing the user with multiple choices and improving the user's experience of using the data analysis associated query system.

[0136] It should be noted that during the data processing process, the alarm module can prompt the user with error information and alarms in a timely manner to ensure the accuracy and integrity of the data. Specifically, by real-time monitoring and prompting of errors in the data processing process, it is convenient for users to find and solve problems in a timely manner. In addition, by providing a data alarm function, it is possible to send alarm information to users in a timely manner when data is abnormal or has potential problems.

[0137] In addition, the data analysis and association query system in the embodiment of the present application can provide a data recovery function, so that users can quickly recover data that was accidentally deleted or misoperated. Specifically, the data analysis and association query system can support data backup and recovery functions, so that users can quickly recover data when data is lost or damaged; and provide data version management functions, so that users can view and restore historical data versions.

[0138] In addition, the data analysis association query system in the embodiment of the present application can effectively manage temporary files in the data analysis process to ensure the stability and security of the system. Specifically, by regularly cleaning and archiving temporary files, it is possible to avoid temporary files from occupying too many system resources. By providing encryption and backup functions for temporary files, the security and integrity of temporary files can be ensured.

[0139] In addition, it should be noted that the data analysis association query system in the embodiment of the present application can adopt a modular design, that is, the data analysis association query system can include a data source selection module corresponding to the "import" control, a data table selection module corresponding to the data table selection area, a field selection module corresponding to the field selection area, an association method selection module corresponding to the association method selection area, an execution module corresponding to the lightweight database, and a result display module. Based on this, by adopting a modular design, it is convenient for users to add new functional modules according to specific needs. In addition, by providing an open interface, it is convenient for third-party developers to integrate and expand the system.

[0140] In summary, the data analysis association query system and the visual data analysis method based on association query in the embodiments of the present application may have the following advantages:

[0141] Lightweight design: specially designed for single-machine environments, no complex configuration required, easy to deploy and use;

[0142] Ease of use: It provides an intuitive and user-friendly visual interface and supports operations such as dragging and clicking, which allows users to quickly build data analysis and related query tasks without requiring users to have a technical background.

[0143] Versatility: Supports multiple data sources, multiple association methods, data visualization and other functions to meet users' diverse data analysis needs.

[0144] Efficiency: It uses efficient data processing algorithms to quickly respond to user query requests, provide instant results, and improve data analysis efficiency.

[0145] Visualization: Display the results of related queries in the form of charts, etc., making the data more intuitive and easy to understand, allowing users to quickly gain insight into key information in the data.

[0146] Based on the visual data analysis method based on association query provided in the above embodiment, correspondingly, the present application also provides a specific implementation of a visual data analysis device based on association query. Please refer to the following embodiment.

[0147] like Figure 5 As shown, the visual data analysis device 500 based on association query provided in the embodiment of the present application includes the following modules:

[0148] A first display module 510 is used to display multiple first data sources in the lightweight database on an interactive interface, where the multiple first data sources are obtained by converting multiple second data sources according to a data format and structure corresponding to the lightweight database, and the multiple second data sources are heterogeneous data sources;

[0149] The first receiving module 520 is used to receive a first input from a user selecting a plurality of target data tables, a plurality of target fields in each target data table, and a target association method from a plurality of first data sources;

[0150] A matching module 530, configured to match multiple target fields selected by a user in response to the first input, and obtain a matching relationship between the multiple target fields;

[0151] A generating module 540, configured to generate a first associated query statement according to a plurality of target data tables, matching relationships and target association modes;

[0152] The first execution module 550 is used to execute the first associated query statement through the lightweight database to obtain and display the first associated query result.

[0153] The above-mentioned visual data analysis device 500 based on association query is described in detail below, as shown below:

[0154] In some embodiments, the matching module 530 may specifically:

[0155] A first determination submodule is used for determining, in response to a first input, for each first target field in a first target data table, a fuzzy matching degree between the first target field and a plurality of target fields in each second target data table, wherein the first target data table is any one of the plurality of target data tables, the first target field is any one of the plurality of target fields in the first target data table, and the second target data table is any one of the plurality of target data tables except the first target data table;

[0156] The second determination submodule is used to determine, for each first target field in the first target data table and each second target data table, the target field corresponding to the fuzzy matching degree with the largest value among multiple fuzzy matching degrees as the matching field of the first target field, so as to obtain the matching relationship between the multiple target fields.

[0157] In some embodiments, the visual data analysis device 500 based on association query may further include:

[0158] A second display module is used to display the matching relationship on the interactive interface after obtaining the matching relationship between the multiple target fields;

[0159] A second receiving module, used to receive a second input from a user for adjusting the matching relationship;

[0160] An updating module is used to update the matching relationship in response to the second input.

[0161] Based on this, the generation module 540 may specifically include:

[0162] The generating submodule is used to generate a first associated query statement according to a plurality of target data tables, the updated matching relationship and the target association mode.

[0163] In some embodiments, the interactive interface includes a data table selection area, a field selection area, and an association mode selection area, and the first input includes a first sub-input, a second sub-input, and a third sub-input. Based on this, the first receiving module 520 may specifically include:

[0164] The receiving submodule is used to receive a first sub-input of a user dragging multiple target data tables from multiple first data sources to a data table selection area, a second sub-input of dragging multiple target fields in each target data table to a field selection area, and a third sub-input of selecting a target association method in an association method selection area.

[0165] In some embodiments, the visual data analysis device 500 based on association query may further include:

[0166] A third display module, used for displaying an interactive interface before the interactive interface displays the plurality of first data sources in the lightweight database;

[0167] A third receiving module, used to receive a third input from a user for importing a plurality of second data sources in the interactive interface;

[0168] a conversion module, configured to, in response to the third input, perform data conversion on the plurality of second data sources according to the data format and structure corresponding to the lightweight database to obtain the plurality of first data sources;

[0169] The import module is used to import multiple first data sources into the lightweight database.

[0170] In some embodiments, the conversion module may specifically include:

[0171] A verification submodule, configured to perform data verification on the data in the plurality of second data sources respectively in response to the third input, and obtain a data verification result;

[0172] The conversion submodule is used to convert the data of the plurality of second data sources according to the data format and structure corresponding to the lightweight database to obtain the plurality of first data sources when the data verification result is that the verification is passed.

[0173] In some embodiments, the interactive interface includes a query statement editing area. Based on this, the visual data analysis device 500 based on association query may also include:

[0174] A fourth receiving module, configured to receive a fourth input of a user editing a second associated query statement in the query statement editing area after the plurality of first data sources in the lightweight database are displayed on the interactive interface;

[0175] A determination module, configured to determine a second associated query statement in response to a fourth input;

[0176] The second execution module is used to execute the second associated query statement through the lightweight database to obtain and display the second associated query result.

[0177] The visual data analysis device based on associative query in the embodiment of the present application performs data analysis by using a lightweight database, without the need for complex installation and configuration processes, and has low technical requirements for data analysts. In the scenario of performing associative query and analysis on multiple data sources, by providing an interactive interface, the user does not need to write complex associative query statements. By selecting multiple target data tables, multiple target fields in each target data table, and target association methods in the interactive interface, the first associative query result can be obtained and displayed, and the technical requirements for data analysts are low. In addition, by automatically generating the first associative query statement based on multiple target data tables, matching relationships, and target association methods, the efficiency of data analysis can be improved compared to the user manually writing the associative query statement.

[0178] Based on the visual data analysis method based on association query provided in the above embodiment, the embodiment of the present application also provides a specific implementation of the electronic device. Figure 6 A schematic diagram of an electronic device 600 provided in an embodiment of the present application is shown.

[0179] The electronic device 600 may include a processor 610 and a memory 620 storing computer program instructions.

[0180] Specifically, the processor 610 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0181] The memory 620 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 620 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. Where appropriate, the memory 620 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 620 may be inside or outside the electronic device 600. In a particular embodiment, the memory 620 is a non-volatile solid-state memory.

[0182] The memory may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk storage medium device, an optical storage medium device, a flash memory device, an electrical, optical or other physical / tangible memory storage device. Thus, typically, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to the first aspect of the present application.

[0183] The processor 610 implements any one of the visualization data analysis methods based on association query in the above embodiments by reading and executing the computer program instructions stored in the memory 620 .

[0184] In one example, the electronic device 600 may further include a communication interface 630 and a bus 640. Figure 6 As shown, the processor 610, the memory 620, and the communication interface 630 are connected via a bus 640 and communicate with each other.

[0185] The communication interface 630 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0186] Bus 640 includes hardware, software or both, and the parts of electronic equipment are coupled to each other. For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 640 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the application considers any suitable bus or interconnection.

[0187] Exemplarily, the electronic device 600 may be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA).

[0188] The electronic device can execute the visual data analysis method based on associated query in the embodiment of the present application, thereby realizing the combination Figures 1 to 5 The described method and device for visual data analysis based on associative query.

[0189] In addition, in combination with the visual data analysis method based on associated query in the above embodiment, the embodiment of the present application can provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by the processor, any one of the visual data analysis methods based on associated query in the above embodiment is implemented.

[0190] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0191] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0192] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0193] The above reference is according to the method of the embodiment of the present application, the flow chart of the device (system) and the computer program product and / or the block diagram described various aspects of the present application.It should be understood that each square box in the flow chart and / or the block diagram and the combination of each square box in the flow chart and / or the block diagram can be realized by computer program instructions.These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the realization of the function / action specified in one or more square boxes of the flow chart and / or the block diagram.Such a processor can be but is not limited to a general-purpose processor, a special-purpose processor, a special application processor or a field programmable logic circuit.It can also be understood that each square box in the block diagram and / or the flow chart and the combination of the square boxes in the block diagram and / or the flow chart can also be realized by the dedicated hardware that performs the specified function or action, or can be realized by the combination of dedicated hardware and computer instructions.

[0194] The above is only a specific implementation of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

Claims

1. A visual data analysis method based on association query, characterized in that: include: Displaying multiple first data sources in a lightweight database on an interactive interface, wherein the multiple first data sources are obtained by converting multiple second data sources according to a data format and structure corresponding to the lightweight database, and the multiple second data sources are heterogeneous data sources; receiving a first input from a user selecting a plurality of target data tables, a plurality of target fields in each of the target data tables, and a target association method from the plurality of first data sources; In response to the first input, matching the multiple target fields selected by the user to obtain a matching relationship between the multiple target fields; Generate a first association query statement according to the multiple target data tables, the matching relationship and the target association mode; The first associated query statement is executed through the lightweight database to obtain and display a first associated query result.

2. The method according to claim 1, characterized in that: In response to the first input, matching the multiple target fields selected by the user to obtain a matching relationship between the multiple target fields includes: In response to the first input, for each first target field in a first target data table, determining a fuzzy matching degree between the first target field and a plurality of target fields in each second target data table, wherein the first target data table is any one of the plurality of target data tables, the first target field is any one of the plurality of target fields in the first target data table, and the second target data table is any one of the plurality of target data tables except the first target data table; For each of the first target fields in the first target data table and each of the second target data table, the target field corresponding to the fuzzy matching degree with the largest value among the multiple fuzzy matching degrees is determined as the matching field of the first target field, so as to obtain the matching relationship between the multiple target fields.

3. The method according to claim 1 or 2, characterized in that: After obtaining the matching relationship between the plurality of target fields, the method further includes: Displaying the matching relationship on the interactive interface; receiving a second input from a user for adjusting the matching relationship; In response to the second input, updating the matching relationship; The generating a first association query statement according to the multiple target data tables, the matching relationship and the target association mode includes: A first association query statement is generated according to the multiple target data tables, the updated matching relationship and the target association mode.

4. The method according to claim 1, characterized in that The interactive interface includes a data table selection area, a field selection area, and an association method selection area, the first input includes a first sub-input, a second sub-input, and a third sub-input; the receiving of the first input from the user selecting multiple target data tables, multiple target fields in each of the target data tables, and target association methods from the multiple first data sources includes: Receive a first sub-input of a user dragging multiple target data tables from the multiple first data sources to the data table selection area, a second sub-input of a user dragging multiple target fields in each of the target data tables to the field selection area, and a third sub-input of a user selecting a target association method in the association method selection area.

5. The method according to claim 1, characterized in that: Before displaying the plurality of first data sources in the lightweight database on the interactive interface, the method further includes: Displaying the interactive interface; receiving a third input from a user in the interactive interface for importing the plurality of second data sources; In response to the third input, performing data conversion on the plurality of second data sources according to a data format and structure corresponding to the lightweight database to obtain the plurality of first data sources; The plurality of first data sources are imported into the lightweight database.

6. The method according to claim 5, characterized in that The step of converting the plurality of second data sources in response to the third input according to the data format and structure corresponding to the lightweight database to obtain the plurality of first data sources includes: In response to the third input, respectively verify the data in the plurality of second data sources to obtain a data verification result; When the data verification result is that the verification is passed, data conversion is performed on the multiple second data sources according to the data format and structure corresponding to the lightweight database to obtain the multiple first data sources.

7. The method according to claim 1, characterized in that The interactive interface includes a query statement editing area. After the interactive interface displays the plurality of first data sources in the lightweight database, the method further includes: receiving a fourth input from a user to edit a second associated query statement in the query statement editing area; In response to the fourth input, determining the second associated query statement; The second associated query statement is executed through the lightweight database to obtain and display a second associated query result.

8. A visual data analysis device based on association query, characterized in that: The device comprises: A first display module is used to display multiple first data sources in the lightweight database on an interactive interface, wherein the multiple first data sources are obtained by converting multiple second data sources according to a data format and structure corresponding to the lightweight database, and the multiple second data sources are heterogeneous data sources; A first receiving module is used to receive a first input from a user selecting a plurality of target data tables, a plurality of target fields in each of the target data tables, and a target association method from the plurality of first data sources; a matching module, configured to match the plurality of target fields selected by the user in response to the first input, and obtain a matching relationship between the plurality of target fields; A generating module, configured to generate a first associated query statement according to the plurality of target data tables, the matching relationship and the target association mode; The first execution module is used to execute the first associated query statement through the lightweight database to obtain and display a first associated query result.

9. An electronic device, characterized in that: The electronic device comprises: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the visual data analysis method based on association query as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, the visual data analysis method based on associative query according to any one of claims 1 to 7 is implemented.

11. A computer program product, characterized in that When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the visual data analysis method based on associative query as described in any one of claims 1 to 7.

Images