Business data analysis method, device, equipment and storage medium
By using user-defined data sets and query SQL to generate materialized views in the analytical database, the problem of low data analysis efficiency under the microservice database architecture is solved, fast and accurate data analysis and interpretation are achieved, and data integration and real-time monitoring are simplified.
Patent Information
- Application Number
- CN202510266141.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-07
- Publication Date
- 2025-09-23
- Estimated Expiration
- 2045-03-07
AI Technical Summary
In the prior art, establishing a separate database for each microservice and analyzing and interpreting the data in each database separately results in low efficiency in data analysis and interpretation.
Query required data in analytical databases using user-defined data sets and query SQL, generate materialized views, and perform data analysis. This avoids analyzing each microservice database separately. OLAP databases and Flink CDC 3.0 are used for data synchronization and real-time processing.
It enables fast and accurate analysis and interpretation of all business data, improves data analysis and interpretation efficiency, simplifies data integration and real-time monitoring, and reduces latency and performance bottlenecks.
Smart Images

Figure CN119760035B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing technology, and in particular to a business data analysis method, device, equipment and storage medium. Background Art
[0002] With the continuous development of Internet technology, the business data generated by enterprises in Internet applications has also increased. The analysis and interpretation of business data is very important to enterprises. Therefore, how to analyze and interpret massive business data in real time, accurately and quickly becomes particularly important.
[0003] In related technologies, when analyzing and interpreting massive amounts of business data, a microservice vertical database architecture is usually established, that is, an independent database is established for each microservice, and then the data in each database is analyzed and interpreted separately.
[0004] However, the above technologies have the problem of low efficiency in data analysis and interpretation. Summary of the Invention
[0005] The present invention provides a business data analysis method, apparatus, equipment and storage medium to address the defect in the prior art of establishing a separate database for each microservice and analyzing and interpreting the data in each database separately, resulting in low efficiency of data analysis and interpretation. The method enables the required data and corresponding data analysis results to be queried and displayed in an analytical database through user-defined data sets and query SQL, eliminating the need to analyze and interpret the data in the database of each microservice, thereby improving the efficiency of data analysis and interpretation.
[0006] The present invention provides a business data analysis method, comprising:
[0007] Obtaining data set configuration information entered by the user on the front-end interface; the data set configuration information includes the definition of the data source and data columns, the data source supports the online analytical processing (OLAP) database, including the newly added first data set, and the data column definition supports filtering conditions and analytical functions;
[0008] Generate a query structured query language SQL based on the definition of the data column, and query the backend analytical database based on the query SQL to determine the target business data corresponding to the first data set and the data analysis results corresponding to the target business data; the above analytical database stores all business data;
[0009] A display interface is generated on the front-end interface; the display interface includes a data display area and / or a query user interface UI, and the data display area displays the data analysis results.
[0010] According to a business data analysis method provided by the present invention, the query is performed in a backend analytical database according to the query SQL to determine the target business data corresponding to the first data set and the data analysis results corresponding to the target business data, including:
[0011] Perform a query in the backend analytical database based on the query SQL to determine the target business data corresponding to the first data set;
[0012] Generates SQL statements for creating materialized views based on target business data, and controls Analytic Database to execute the SQL statements to generate the target materialized views corresponding to the target business data.
[0013] Perform a select operation on the target materialized view to determine the data analysis results corresponding to the target business data.
[0014] According to a business data analysis method provided by the present invention, the data set configuration information further includes a materialization type, and the materialization type includes synchronous materialization or asynchronous materialization. The method further includes:
[0015] If the materialization type is asynchronous materialization, obtain the refresh interval and bucket fields entered by the user on the front-end interface;
[0016] The updated data corresponding to the bucketed fields in the first data set are periodically queried in the analytical database according to the refresh interval, and the target materialized view is updated according to the updated data.
[0017] According to a business data analysis method provided by the present invention, the data set configuration information further includes a materialization type, and the step of generating a materialized view based on target business data and creating SQL includes:
[0018] Generates SQL statements for creating a materialized view that matches the materialized type based on the target business data and the materialized type. Different materialized types have different SQL statements for creating a materialized view.
[0019] According to a business data analysis method provided by the present invention, the method further includes:
[0020] Obtaining deletion information input by a user on a front-end interface; the deletion information includes a second data set selected by the user from a data set list on the front-end interface and a deletion operation to be performed on the second data set, wherein the data set list includes at least one data set;
[0021] Detecting whether the second data set has been used; the above-mentioned use includes that the second data set has generated an analysis list or an analysis chart, or the second data set is generating an analysis list or an analysis chart;
[0022] If the second data set has been used, a first prompt message is output; the first prompt message is used to indicate that the second data set cannot be deleted.
[0023] According to a business data analysis method provided by the present invention, the method further includes:
[0024] If the second data set is not used, delete the second data set from the data set list and execute the materialized view deletion SQL.
[0025] According to a business data analysis method provided by the present invention, the method further includes:
[0026] Receive a chart setting operation input by a user and, in response to the chart setting operation, display a chart setting interface; the chart setting interface includes a first tab interface and a second tab interface; the first tab interface is an interface for setting data required for creating a chart, and the second tab interface is an interface for setting styles required for creating a chart;
[0027] Acquiring data setting information entered by a user on a first tab interface and style setting information entered on a second tab interface; the data setting information includes at least one of a third data set selected in the data set list, selected fields for the X-axis and the Y-axis, and set specific parameters of the chart; and the style setting information includes attribute information of the chart;
[0028] An analysis chart corresponding to the third data set is generated according to the data setting information and the style setting information.
[0029] According to a business data analysis method provided by the present invention, the method further includes:
[0030] Execute data migration commands and use the data integration tool Flink CDC 3.0 to obtain data tables from different source databases in real time;
[0031] Create a data table in Analytic Database that has the same table structure as the data table in each source database, and synchronously migrate each data table in each source database to the data table in the corresponding table structure in Analytic Database.
[0032] The present invention also provides a business data analysis device, comprising the following modules:
[0033] The configuration information acquisition module is used to obtain the data set configuration information entered by the user on the front-end interface; the above data set configuration information includes the definition of the data source and data column. The data source supports OLAP database, including the newly added first data set. The definition of the data column supports filtering conditions and analysis functions.
[0034] A query module, configured to generate query SQL based on the definition of the data column, and perform a query in a backend analytical database based on the query SQL to determine the target business data corresponding to the first data set and the data analysis results corresponding to the target business data; the analytical database stores all business data;
[0035] The analysis result display module is used to generate a display interface on the front-end interface; the above display interface includes a data display area and / or a query UI, and the data display area displays the data analysis results.
[0036] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the business data analysis method as described above is implemented.
[0037] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the business data analysis methods described above.
[0038] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the business data analysis method as described above is implemented.
[0039] The business data analysis method, device, equipment, and storage medium provided by the present invention obtain data set configuration information input by a user on a front-end interface, generate query SQL based on the definition of the data column in the data set configuration information, and query in the back-end analytical database according to the query SQL to determine the target business data corresponding to the first data set newly added in the data source included in the data set configuration information and the data analysis results corresponding to the target business data, and generate a display interface on the front-end interface, the display interface including a data display area and / or a query UI, the data display area displays the data analysis results; wherein, the analytical database stores all business data, the data set configuration information includes the definition of the data source and the data column, the data source supports the OLAP database, and the data column definition supports filtering conditions and analysis functions. In this method, after the required data is queried in the analytical database through the user-defined data set and query SQL, data analysis is performed to obtain the corresponding data analysis results and display them, thereby realizing rapid analysis and interpretation of all business data through the analytical database and query SQL, without the need to analyze and interpret the data in the database of each microservice, thereby improving the efficiency of data analysis and interpretation. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0041] Figure 1 It is a flowchart of the business data analysis method provided by the present invention.
[0042] Figure 2 This is a diagram of the ELT model based on Flink CDC 3.0 and StarRocks provided by the present invention.
[0043] Figure 3 This is a schematic diagram of the diverse visualization integration provided by the present invention.
[0044] Figure 4 It is a structural diagram of the business data analysis device provided by the present invention.
[0045] Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION
[0046] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.
[0047] Currently, the importance of business data is becoming increasingly significant. Faced with rapidly evolving market demands, enterprises urgently need to analyze and interpret massive amounts of data in real time, quickly, and accurately. This pursuit of real-time performance is not only about responding to market dynamics but also about providing a solid data foundation for decision-making and business optimization. Especially in the context of rapidly surging data volumes, enterprises must be able to process and analyze large datasets to gain deep insights and quickly identify potential opportunities or risks. This reliance on real-time data analysis enables enterprises to maintain agility in a highly competitive market, leading to greater efficiency and accuracy in business decision-making. In enterprise applications employing a microservices-based vertical sharding architecture, sharding improves system scalability and flexibility, but also presents new challenges and opportunities for data statistical analysis. With the rapid development of enterprise businesses and the iteration of personalized software features, data management, integration, and analysis have become increasingly important. However, microservices-based sharding architectures commonly suffer from the following drawbacks in terms of real-time and large-scale data analysis: 1. Isolated data storage: Each microservice maintains its own independent database, making data integration difficult and creating data silos. This complicates cross-service business analysis, requiring, for example, manual aggregation of data from different services, which is error-prone and time-consuming. 2. Complex ETL (Extract, Transform, Load) Processes: Comprehensive data analysis requires complex ETL processes, which not only increases development and maintenance workloads but can also lead to data latency. 3. Latency Issues: In a microservices architecture, data transfer and transformation between services can introduce latency, impacting real-time analysis capabilities. This is particularly true in event-driven architectures, where event processing, delivery, and aggregation often require time, potentially delaying decision-making. 4. Difficulty in Real-Time Monitoring: In a rapidly changing business environment, real-time monitoring and analysis are crucial. However, decentralized data storage and processing architectures make it difficult to capture and analyze real-time data, hindering the ability to quickly respond to business dynamics. 5. Performance Bottlenecks: When processing large amounts of data, distributed databases often require complex queries and aggregations, which can lead to performance bottlenecks. In a microservices architecture, multiple microservices simultaneously accessing their own databases for complex analysis can result in a large number of I / O (Input / Output) operations, impacting overall system performance. 6. Data storage and query efficiency: After database sharding, the data in each database is unevenly distributed, and the data volume of some services may surge. For the analysis of such large amounts of data, it is usually necessary to optimize the storage and query methods. However, in an independent database environment, this optimization work becomes complicated and difficult to manage uniformly.
[0048] Based on this, embodiments of the present invention provide a business data analysis method, apparatus, device, and storage medium, which can solve this technical problem.
[0049] The following combination Figure 1-Figure 3 The business data analysis method according to an embodiment of the present invention is described.
[0050] It should be noted that the execution subject of the embodiments of the present invention can be a business data analysis device, an electronic device, or other devices or equipment. The following embodiments will be described using an electronic device as an example of the execution subject.
[0051] Figure 1 It is a flowchart of the business data analysis method provided by the present invention, such as Figure 1 As shown, the method includes the following steps:
[0052] S102, obtaining data set configuration information input by the user on the front-end interface; the above data set configuration information includes the definition of the data source and data columns, the data source supports the online analytical processing OLAP database, including the newly added first data set, and the data column definition supports filtering conditions and analysis functions.
[0053] The electronic device may include a dynamic data set management module for managing data sets. The dynamic data set management module may include a front-end interface that provides an entry for user input of information. The front-end interface serves as an interface for interaction between the front-end of the dynamic data set management module and the back-end / server.
[0054] In addition, the front-end interface can display data sources in the form of a dataset list. The dataset list can display one or more datasets and related information for each dataset. The relevant information for each dataset may include the dataset name, creation time, dataset status, and operations. The dynamic dataset management module can include dataset list management, dataset editing, adding, and deleting. The dataset list management primarily manages the datasets in the dataset list displayed on the front-end interface.
[0055] To customize a data source / dataset, users can click the "Add Dataset" button on the front-end interface to enter the dataset creation interface / page. On this interface / page, users can enter dataset configuration information, including the desired data source and data column definitions. The data source can include the desired dataset and its related information. For example, the desired dataset can include a newly added dataset, referred to as the first dataset. After entering this dataset configuration information, relevant information about the first dataset (such as its view name, dataset name, and dataset description) is obtained. At this point, the first dataset is empty, and the following steps are required to retrieve the required data for the first dataset. After obtaining the newly added first dataset, it can be added to the dataset list, which will update the displayed dataset list. Furthermore, this data source supports Online Analytical Processing (OLAP) databases. OLAP databases are used for data analysis and decision support, primarily for querying and analyzing multidimensional data. They allow users to perform complex queries and analyses based on various dimensions (such as time, region, and product), making them suitable for aggregate calculations on large-scale data. The OLAP databases here may include ClickHouse, BigQuery, StarRocks, etc.
[0056] The data column definitions in the dataset configuration information above support filtering conditions and analysis functions. Filtering conditions refer to the conditions for filtering columns in a database table, and you can define the specific filtering conditions. Analysis functions refer to functions that analyze columns in a database table, such as sum analysis, difference analysis, and weighted average analysis.
[0057] S104, generate a query structured query language SQL according to the definition of the data column, and query in the back-end analytical database according to the query SQL to determine the target business data corresponding to the first data set and the data analysis results corresponding to the target business data; all business data are stored in the above analytical database.
[0058] After obtaining the data column definitions in the dataset configuration information, matching query SQL can be generated based on the filtering conditions and analysis functions in the data column definitions. Query SQL here refers to the SQL that queries the data source (e.g., the first dataset) based on the filtering conditions and analysis functions in the data column definitions. For example, this query SQL can be used to filter out certain data in the database or analyze certain data. SQL refers to Structured Query Language, and this query SQL can be used to query the corresponding data to construct the dataset.
[0059] After generating the query SQL, the backend analytical database can be queried based on the query SQL to obtain the desired data analysis results. As an optional embodiment, during the query, the backend analytical database can be queried based on the query SQL to determine the target business data corresponding to the first data set; a materialized view creation SQL can be generated based on the target business data, and the analytical database can be controlled to execute the materialized view creation SQL to generate a target materialized view corresponding to the target business data; a selection operation can be performed on the target materialized view to determine the data analysis results corresponding to the target business data.
[0060] Specifically, when querying the data required by a data source (such as the first data set), the query may be performed in an analytical database (such as a StarRocks database) that stores all business data. Taking querying business data in the StarRocks database as an example, before querying the business data in the StarRocks database, the business data needs to be synchronized to the StarRocks database. The synchronization method for the business data stored in the StarRocks database may optionally include:
[0061] Execute data migration commands and use the data integration tool Flink CDC 3.0 to obtain data tables from different source databases in real time. Create data tables in Analytic Database (such as StarRocks) with the same table structure as the data tables in each source database, and synchronously migrate each data table in each source database to a data table with the corresponding table structure in Analytic Database (such as StarRocks).
[0062] For example, take querying business data in the StarRocks database as an example. Figure 2The ELT model diagram based on Flink CDC3.0 and StarRocks is shown. The business data was originally stored in different source MySQL / SQL databases (only MySQL is used for explanation below). In this embodiment, the data processing module in the electronic device can be used to synchronously migrate all business data stored in different source MySQL databases to the same StarRocks. StarRocks here is a high-performance analytical database designed for online analytical processing scenarios. It can process large-scale data sets and support real-time data query and analysis. In addition, the data processing module here uses a real-time data acquisition tool based on FlinkCDC 3.0 when synchronously migrating data. At the same time, combined with StarRocks' ELT (Extract, Load, Transform) data processing method, real-time and efficient data synchronization can be achieved.
[0063] The specific synchronization and migration of business data can be divided into two parts: one is the synchronization of library and table structures, and the other is the real-time synchronization of data.
[0064] For synchronizing the library and table structures, the data processing module can use the data migration tool "**StarRocks Migration Tool(SMT)**" (also known as StarRocks-migrate-tools). The specific migration process is: according to the configuration information of the source MySQL and StarRocks, execute the command ". / starrocks-migrate-tool" (that is, the command to start the above data migration tool, which is used to migrate the database to StarRocks), read the library and table structure to be synchronized from the source MySQL, and generate an SQL file (starrocks-create.all.sql). This SQL file is used to create the corresponding target library and table in the StarRocks analysis library, thereby ensuring the consistency of the table structure in the source database and the analytical database. After executing the command to start the above data migration tool, the data processing module can connect to StarRocks and execute the data migration command: copycodesql mysql -h<fe_host> -P<fe_query_port> -u user2 -p <password><starrocks-create.all.sql. The meanings of the commands in this data migration command are as follows: copy codesql: Indicates using the copy command to copy each SQL text to StarRocks. mysql: Specifies the transfer method as the MySQL protocol, usually used for connecting to StarRocks. -h <fe_host>: Specifies the hostname or IP (Internet Protocol) address of the Frontend (FE) node. -P <fe_query_port>: Specifies the query port of the Frontend node, usually 9030 (may vary according to the configuration). -u user2: Specifies the username used to connect to StarRocks. -p <password>:Specify the password of the user. Note that there is no space before the password. <starrocks-create.all.sql>: Specify the SQL script file to be executed, which contains SQL statements for creating databases, tables, and related objects. By repeatedly executing the above data migration commands, the table structures in the source MySQL databases of different microservices that need to be synchronized can be synchronized to the analytical database StarRocks.
[0065] For real-time data synchronization, Flink CDC 3.0 provides real-time data capture capabilities. The data processing module can use the Flink CDC Connector (a connection component for Flink data collection tools) to connect to the source data tables (Source table) in the source MySQL, read the binary log file Binlog of the source MySQL in real-time, and monitor the change events (such as insert, update, delete, etc.) in the data tables of the source MySQL. Whenever the data in the source MySQL changes, Flink CDC 3.0 can immediately detect it and generate corresponding pipeline / stream data for the changed data. Then, the SinkConnector component of Flink (a component for writing data to external systems) can be used to insert the generated pipeline / stream data into the corresponding target data tables (Sink table) in StarRocks through high concurrency. StarRocks and the Sink table are connected through the starrocks flink-connector (a connection component for starrocks data collection tools). In addition, the data processing module can configure / add their respective Flink Jobs (i.e., data collection tasks) for the source MySQL of different microservices, and by executing the Flink Jobs of each source MySQL respectively, achieve real-time parallel monitoring of data changes in each source MySQL, and update the updated data in each MySQL to StarRocks in real-time parallel, realizing real-time data synchronization for multiple data sources / multiple source databases after vertical database sharding of microservices.
[0066] Through data synchronization within the aforementioned data processing module, data from different source databases can be simultaneously migrated to a single analytical database (such as a StarRocks database), effectively storing all business data within the analytical database. After a user enters the dataset configuration information on the front-end interface to obtain the name of a first dataset, they can click the "Parse and Run" button on the front-end interface. This notifies the electronic device backend that data analysis is required for the first dataset. The generated query SQL can then be executed within the analytical database. If the query SQL successfully executes, the business data corresponding to the first dataset is obtained and recorded as the target business data. If the query SQL fails or fails to execute, a specific error message can be displayed in the data preview area (or data display area), detailing the cause of the query SQL execution error, such as a query SQL configuration error. This allows the user to quickly identify the cause of the query SQL execution error and promptly make changes to ensure successful query SQL execution.
[0067] After obtaining the target business data required for data analysis from the first dataset in the Analytic Database, you can perform data analysis on the target business data. This analysis specifically includes processing and analyzing the materialized view of the target business data. This materialized view processing and analysis can be performed by first generating SQL to create the materialized view and then executing this SQL in the Analytic Database. This involves attempting to create the materialized view using the target business data in the Analytic Database. If the SQL for creating the materialized view of the target business data executes successfully, the corresponding materialized view can be generated. The generated materialized view of the target business data can be recorded as the target materialized view.
[0068] After the materialized view of the target business data is successfully created, you can perform a SELECT operation on the materialized view, that is, interpret the analysis results of the target materialized view to obtain the corresponding data analysis results, that is, obtain the data analysis results corresponding to the target business data.
[0069] S106, generating a display interface on the front-end interface; the display interface includes a data display area and / or a query user interface UI, and the data display area displays the data analysis results.
[0070] In this step, after obtaining the data query results corresponding to the target business data, a display interface can be automatically generated in the front-end interface. This display interface may include a data display area, a query UI interface, and other areas. The data display area can display the data analysis results obtained after the above data query and analysis, and the query UI interface can display query controls / buttons provided to the user for query use.
[0071] The positions of the data display area and the query UI interface on the front-end interface can be set according to actual conditions and are not specifically limited here.
[0072] In addition, when displaying the data analysis results of the target business data in the data display area, it can be displayed in the form of a paging list, which makes it convenient for users to check the data analysis results of different data sets.
[0073] Furthermore, the above-mentioned analysis results of the target materialized view are interpreted to obtain data analysis results of the target business data. Specifically, the target materialized view can be disassembled, such as the meaning of the horizontal and vertical axes of the curves included in the target materialized view, the range of the horizontal and vertical axes, the meaning of the curves or graphics included in the target materialized view, etc. This is easier for users to understand, thereby improving the efficiency of data analysis.
[0074] In this embodiment, by obtaining the data set configuration information input by the user on the front-end interface, generating a query SQL based on the definition of the data column in the data set configuration information, and querying the analytical database on the back-end according to the query SQL, the target business data corresponding to the first data set newly added in the data source included in the data set configuration information and the data analysis results corresponding to the target business data are determined, and a display interface is generated on the front-end interface, the display interface includes a data display area and / or a query UI, and the data display area displays the data analysis results; wherein, all business data is stored in the analytical database, the data set configuration information includes the definition of the data source and the data column, the data source supports the OLAP database, and the definition of the data column supports filtering conditions and analysis functions. In this method, after the required data is queried in the analytical database through the user-defined data set and query SQL, data analysis is performed to obtain the corresponding data analysis results and display them, thereby realizing the rapid analysis and interpretation of all business data through the analytical database and query SQL, without the need to analyze and interpret the data in the database of each microservice, thereby improving the efficiency of data analysis and interpretation.
[0075] The following embodiment describes a specific process of updating a target materialized view when the data set configuration information further includes a materialization type, and the materialization type includes synchronous materialization or asynchronous materialization.
[0076] In some embodiments, the above method may further include the following steps:
[0077] Step A1: If the materialization type is asynchronous materialization, obtain the refresh interval and bucket fields entered by the user on the front-end interface.
[0078] Among them, when users enter the data set configuration information, they can also select the required materialization type in the materialization type option on the front-end interface. The materialization type refers to the synchronization type when generating the materialized view. The materialization type generally includes synchronous materialization and asynchronous materialization. Synchronous materialization refers to the real-time generation of materialized views based on the business data in the analytical database (such as the StarRocks database). That is, when the business data in the analytical database changes, the materialized views generated based on the business data in the analytical database will also be updated synchronously in real time. Asynchronous materialization means that when the business data in the analytical database changes, the materialized views generated based on the business data in the analytical database will only be updated when certain conditions are met, rather than in real time. This ensures that the materialized views currently being used by users are not affected, thereby improving the user experience.
[0079] This embodiment primarily describes the update process for a materialized view using asynchronous materialization. When a user selects asynchronous materialization, the front-end interface displays an interface or option requiring the user to enter a refresh interval and bucket fields. The user can enter the desired refresh interval and bucket fields to query using this interface or option. The refresh interval refers to the update interval or time for the generated materialized view, such as 5 minutes or 10 minutes. Bucket fields refer to the fields in the target business data corresponding to the first dataset in the analytical database that are used for data query.
[0080] Step A2: Query the analytical database for updated data corresponding to the bucketed fields in the first dataset periodically according to the refresh interval, and update the target materialized view according to the updated data.
[0081] In this step, after obtaining the refresh interval and bucket fields entered by the user, the system monitors whether the time interval between the current time and the last time the target materialized view was generated / updated reaches the refresh interval. If so, the system then finds the corresponding updated data in the first dataset in AnalyticDB based on the bucket fields. This updated data is then used to update the previously generated / updated target materialized view, obtaining the updated target materialized view. This query based on the bucket fields avoids full-table queries, improving query efficiency.
[0082] In an optional embodiment, after the user enters a materialization type, the "Generate Materialized View Creation SQL Based on Target Business Data" step can include generating materialized view creation SQL matching the target business data and the materialization type. Different materialization types correspond to different materialized view creation SQL statements. In other words, if the user enters a materialization type, the corresponding materialized view creation SQL statement can be generated based on the entered materialization type. This improves the accuracy of the generated materialized view creation SQL statement, as well as the accuracy and success rate of materialized view creation.
[0083] In this embodiment, when the user selects asynchronous materialization, the refresh interval and bucket fields entered by the user are used to retrieve corresponding updated data from the Analytic Database using the bucket fields when the refresh interval is reached. This updates the previously generated / updated target materialized view. This ensures that the previously generated materialized view remains functional within a certain timeframe, thereby improving the user experience. Furthermore, the corresponding materialized view creation SQL is generated based on the materialization type entered by the user, improving the accuracy of the generated materialized view creation SQL and the accuracy and success rate of the final materialized view generation.
[0084] The following embodiment describes a process of successfully deleting a dataset in a dataset list that can be displayed on a front-end interface and can include relevant information of one or more datasets, and when a user needs to delete a dataset in the dataset list.
[0085] In some embodiments, the above method may further include the following steps:
[0086] Step B1, obtaining deletion information input by the user on the front-end interface; the deletion information includes a second dataset selected by the user from a dataset list on the front-end interface and a deletion operation to be performed on the second dataset, wherein the dataset list includes at least one dataset.
[0087] Among them, as explained above, the functions of the dynamic dataset management module include the dataset deletion function, that is, this embodiment provides the function of deleting datasets in the dataset list. The dataset list may include one or more datasets, which may be datasets added by users through new functions, or datasets automatically generated by data in the analytical database, or may also include datasets obtained by other means.
[0088] When a user needs to delete a data set in the data set list, the user can enter deletion information on the front-end interface. The deletion information here can be, for example, the user selecting the data set to be deleted in the data set list, recorded as the second data set, and then performing a trigger operation such as clicking on the selected data set. The front-end interface can display the operations that can be performed on the data set, and the user can then select the desired deletion operation from these operations. In this way, the electronic device can obtain the deletion operation that the user wants to delete the second data set.
[0089] Step B2: Detect whether the second data set has been used. The above-mentioned use includes that the second data set has generated an analysis list or analysis chart, or the second data set is generating an analysis list or analysis chart.
[0090] In this step, after the user selects the second data set to be deleted, it is possible to detect whether the second data set is being generated or has generated a corresponding analysis list or analysis chart, that is, to determine whether the second data set and its corresponding data have been used, and obtain the corresponding detection result or judgment result.
[0091] Step B3: If the second data set has been used, output a first prompt message; the first prompt message is used to indicate that the second data set cannot be deleted.
[0092] In this step, if the second dataset is being generated or has already generated a corresponding analysis list or analysis chart, this indicates that the second dataset has been used in the subsequent chart / list analysis process. To ensure that the user can still use the generated analysis chart or analysis list normally, the second dataset cannot be deleted. In this case, a first prompt message may be displayed to inform the user that the second dataset cannot be deleted. Of course, the reason why the second dataset cannot be deleted may also be displayed so that the user is promptly informed of the reason and the user experience is guaranteed.
[0093] In addition, as an optional embodiment, if the second dataset is unused, the second dataset is deleted from the dataset list and the materialized view drop SQL is executed. In other words, if the second dataset does not generate a corresponding analysis list or analysis chart, it indicates that the second dataset is not used in the subsequent chart / list analysis process. In this case, the second dataset and its related information can be directly deleted from the dataset list. At the same time, the materialized view generated by the second dataset can be deleted by executing the materialized view drop SQL (DROPMATERIALIZED VIEW SQL), thereby avoiding errors in subsequent data queries and saving storage space.
[0094] In this embodiment, when a user wishes to delete a dataset, the system first determines whether the dataset is in use. If so, a prompt message is displayed to inform the user that the dataset cannot be deleted. This ensures that the user can still use the analysis charts and analysis lists generated from the dataset, improving the user experience. Furthermore, if the dataset is unused, the dataset and its corresponding materialized view can be deleted. This prevents errors in subsequent data queries and saves storage space.
[0095] To facilitate better analysis and presentation of data in dataset lists, the electronic device in embodiments of the present invention also includes a visualization module. This module provides a complete front-end and candidate / server interface, enables the definition of analysis lists and chart pairs, and supports a variety of visualizations. Specific functions provided include chart list management, chart creation, preview, design, editing, copying, deleting, and publishing. The following examples illustrate the chart or list design / creation process within this visualization module.
[0096] In some embodiments, the above method may further include the following steps:
[0097] Step C1, receiving a chart setting operation input by a user, and displaying a chart setting interface in response to the chart setting operation; the chart setting interface includes a first tab interface and a second tab interface; the first tab interface is a data setting interface required for creating a chart, and the second tab interface is a style setting interface required for creating a chart.
[0098] In this step, we first explain the chart list management function of the visualization display module. The chart list uses card-tiled thumbnails to display all created visualization charts in the front-end interface. It can also obtain relevant data of the chart or list from the back-end through the API (Application Programming Interface), display the name and thumbnail of the chart or list, and provide operation controls (such as operation buttons) such as preview, design, edit, delete, and copy.
[0099] Secondly, for the chart creation function, after the user clicks the icon to create the related control of the new function, a chart configuration page / interface can pop up. The user can enter the chart name, sorting, remarks and other information on the chart configuration page / interface and submit the input data. After submission, the input data can be sent to the backend through the API interface. The candidate can save the configuration information related to the chart and return a success status. The chart list can automatically refresh and display the newly added chart.
[0100] For the chart design function, users can click on the relevant controls of the chart design on the front-end interface to enter the chart design interface. The left side of the chart design interface can display a variety of commonly used graphic reports (such as tables, line charts, bar charts, pie charts, scatter plots, water wave charts, funnel charts, etc.). Users can select the required graphics from the various commonly used graphic reports and drag them to the design area on the right side of the chart design interface for subsequent processing. It should be noted that it supports dragging and displaying multiple charts in the same chart design interface. These multiple charts can be the same chart or different charts, which can increase the number of chart analyses and thus improve the efficiency of chart analysis and decision-making.
[0101] After one or more chart graphics are displayed in the right design area of the above-mentioned chart design interface, taking a chart graphic as an example, the user can click on the chart graphic in the right design area of the chart design interface. The click operation represents the setting operation of the chart, so that the electronic device can obtain the chart setting operation input by the user. Afterwards, the electronic device can respond to the chart setting operation and pop up a chart setting interface in the right area. The chart setting interface includes two tabs. Different tabs correspond to different setting interfaces. The setting interfaces corresponding to the two tabs can be respectively referred to as the first tab interface and the second tab interface. The first tab interface is the data setting interface required to create the chart, and the second tab interface is the style setting interface required to create the chart.
[0102] Step C2, obtaining data setting information entered by the user on the first tab interface and style setting information entered on the second tab interface; the above-mentioned data setting information includes at least one of the third data set selected in the data set list, the selected fields for the X-axis and the Y-axis, and the set specific parameters of the chart; the above-mentioned style setting information includes attribute information of the chart.
[0103] In this step, the first tab interface includes a variety of data setting options, including dataset settings, data configuration, and parameter configuration. In the dataset settings option, the user can select a dataset from the existing datasets in the dataset list as the data source for the analysis chart. For example, the selected dataset can be recorded as the third dataset. In the data configuration option, the user can select the fields for the X-axis and Y-axis of the analysis chart, that is, the meaning of the fields represented by the X-axis and Y-axis respectively; in addition, the stacked group field and the Chinese interpretation of the table field can be set. In the parameter configuration option, the user can set specific parameters for the chart, which can include search conditions, fixed conditions, and so on.
[0104] The second tab interface includes information for setting the style of the chart. The style setting related information here mainly includes the basic attribute information of the chart, such as the title text, font, color, etc.; at the same time, different chart types can provide their own unique setting options. For example, for a pie chart, you can set the fill color of the pie chart, and for a line chart, you can set the line color of the line chart, etc.
[0105] The above-mentioned first tab page and second tab interface can be jumped to different tab interfaces by the user clicking different tabs on the chart setting interface to perform corresponding data settings and style settings. After the user sets the data and style on the corresponding interface, the electronic device can obtain the corresponding data setting information and style setting information of the chart.
[0106] In addition, when the user sets the relevant data and style of the chart on the corresponding interface, the preview area can update the displayed chart in real time to ensure that the user can see the modified chart effect in a timely and rapid manner.
[0107] Step C3: Generate an analysis chart corresponding to the third data set according to the data setting information and the style setting information.
[0108] In this step, after obtaining the data setting information and style setting information corresponding to the chart at the front end of the electronic device, the data setting information and style setting information corresponding to the chart can be sent to the back end through the API interface. The back end saves the data setting information and style setting information corresponding to the chart, and synchronously generates a chart thumbnail corresponding to the third data set based on this, which is recorded as an analysis chart.
[0109] Furthermore, users can view the visualization of the generated analysis chart in a new window through the "Preview" button on the front-end interface to ensure that the configuration is correct. At the same time, you can click the "Edit" button on the front-end interface to modify the information of the analysis chart, and click the "Copy" button to copy all the configurations of the current analysis chart to the new chart to support independent design. In addition, you can also click the "Delete" button to delete the generated analysis chart. Here, a confirmation box can pop up when deleting the analysis chart. The confirmation box can include a message reminding the user that the deletion operation is irreversible. If the user confirms the deletion, the front-end can call the API interface to perform the deletion operation. After the deletion operation is successful, the chart will be removed from the chart list. At the same time, the chart list and the display content of the front-end interface can be updated.
[0110] For example, see Figure 3 The diverse visualization integration diagram shown in the figure can include a variety of different styles of charts, and the charts can display the meaning of the horizontal and vertical axes, the colors of the charts and other information, which can help users understand the meaning behind the charts and improve decision-making efficiency and accuracy. Figure 3 It is only an example, and the specific content displayed therein (such as specific text or specific numbers, etc.) does not affect the technical essence of the embodiments of the present invention.
[0111] In this embodiment, a rich variety of chart options are provided to users through the visualization display module, which makes it easy for users to customize the required chart information according to their needs and generate corresponding analysis charts, thereby realizing the user's personalized chart generation needs; at the same time, multiple different setting options can be provided for users to choose from, which makes it easy for users to quickly understand the information behind the data and improve the work efficiency and satisfaction of analysts.
[0112] As can be seen from the preceding examples, the present invention utilizes Flink CDC 3.0 to achieve real-time data capture and synchronization, improving the timeliness of data analysis. Furthermore, leveraging the column-based storage and optimized query engine of analytical databases (such as StarRocks), and employing both synchronous and asynchronous materialization, it addresses the efficiency challenges of massive data analysis queries. Users can also customize dynamic datasets based on their needs, flexibly addressing diverse analysis scenarios. Furthermore, a rich selection of chart options provides users with a wider range of visualization options, enabling them to quickly understand the underlying data.
[0113] The technical solutions of the embodiments of the present invention have the following technical effects:
[0114] 1. Improve decision-making efficiency: Through real-time data analysis and visualization, the system enables companies to quickly gain market insights, thereby optimizing the decision-making process, reducing decision-making time, and improving the accuracy and effectiveness of decisions.
[0115] 2. Enhance data integration capabilities: Effectively integrate data from different microservices, reduce data silos, make cross-departmental and cross-business analysis simpler and more efficient, and provide enterprises with a comprehensive business perspective.
[0116] 3. Optimize user experience: The user-friendly front-end interactive interface and flexible dataset management functions enable users to easily create and manage datasets and visualization charts, improving analysts' work efficiency and satisfaction.
[0117] 4. Improve analysis accuracy: By using the high-performance query engine and materialized views of the analytical database, errors caused by query complexity in large-scale data analysis can be reduced, ensuring data accuracy and consistency, thereby enhancing the credibility of the results.
[0118] 5. Reduce operating costs: Through optimized ETL processes and real-time data processing capabilities, the system reduces the manpower and time costs required for data processing and maintenance, while improving resource utilization efficiency and reducing overall operating costs for the enterprise.
[0119] 6. Support diverse data presentation: Provide a variety of visual chart formats so that users with different backgrounds and needs can quickly understand and apply data analysis results, promoting the spread of data-driven decision-making culture within the enterprise.
[0120] The business data analysis device provided by the present invention is described below. The business data analysis device described below and the business data analysis method described above can be referenced to each other.
[0121] Figure 4 This is a schematic diagram of the structure of the business data analysis device provided by the present invention, see Figure 4 As shown, the device may include:
[0122] Configuration information acquisition module 410 is used to acquire data set configuration information entered by the user on the front-end interface; the data set configuration information includes the definition of the data source and data columns. The data source supports OLAP databases, including the newly added first data set. The data column definition supports filtering conditions and analysis functions.
[0123] Query module 420, configured to generate query SQL based on the definition of the data column, and perform a query in a backend analytical database based on the query SQL to determine the target business data corresponding to the first data set and the data analysis results corresponding to the target business data; the analytical database stores all business data;
[0124] The analysis result display module 430 is used to generate a display interface on the front-end interface; the display interface includes a data display area and / or a query UI, and the data display area displays the data analysis results.
[0125] In some embodiments, the above-mentioned query module 420 is specifically used to query in the back-end analytical database according to the query SQL to determine the target business data corresponding to the first data set; generate a materialized view creation SQL according to the target business data, and control the analytical database to execute the materialized view creation SQL to generate a target materialized view corresponding to the target business data; perform a selection operation on the target materialized view to determine the data analysis results corresponding to the target business data.
[0126] In some embodiments, the dataset configuration information further includes a materialization type, where the materialization type includes synchronous materialization or asynchronous materialization. The apparatus further includes:
[0127] The interval and field acquisition module is used to obtain the refresh interval and bucket fields entered by the user on the front-end interface if the materialization type is asynchronous materialization;
[0128] The update module is used to query the analytical database for updated data corresponding to the bucketed fields in the first data set at regular intervals according to the refresh interval, and update the target materialized view according to the updated data.
[0129] Optionally, the data set configuration information further includes a materialization type. The query module 420 is specifically configured to generate a materialized view creation SQL that matches the materialization type based on the target business data and the materialization type. Different materialization types correspond to different materialized view creation SQLs.
[0130] In some embodiments, the apparatus further comprises:
[0131] a deletion information acquisition module, configured to acquire deletion information input by a user on a front-end interface; the deletion information includes a second data set selected by the user from a data set list on the front-end interface and a deletion operation to be performed on the second data set, wherein the data set list includes at least one data set;
[0132] a detection module, configured to detect whether the second data set has been used; the above-mentioned use includes that the second data set has generated an analysis list or analysis chart, or the second data set is generating an analysis list or analysis chart;
[0133] The output module is configured to output a first prompt message if the second data set has been used; the first prompt message is used to indicate that the second data set cannot be deleted.
[0134] Optionally, the apparatus further comprises a deletion module, configured to delete the second data set from the data set list and execute a materialized view deletion SQL if the second data set is not used.
[0135] In some embodiments, the apparatus further comprises:
[0136] A display module is configured to receive a chart setting operation input by a user and, in response to the chart setting operation, display a chart setting interface; the chart setting interface includes a first tab interface and a second tab interface; the first tab interface is a data setting interface required for creating a chart, and the second tab interface is a style setting interface required for creating a chart;
[0137] a setting information acquisition module for acquiring data setting information entered by a user on the first tab interface and style setting information entered on the second tab interface; the data setting information includes at least one of the third data set selected in the data set list, the selected fields for the X-axis and the Y-axis, and the set specific parameters of the chart; and the style setting information includes attribute information of the chart;
[0138] The chart generating module is used to generate an analysis chart corresponding to the third data set according to the data setting information and the style setting information.
[0139] In some embodiments, the apparatus further comprises:
[0140] The synchronous migration module is used to execute data migration commands and use the data integration tool Flink CDC 3.0 to obtain data tables from different source databases in real time. It creates data tables in Analytic Database with the same table structure as the data tables in each source database and synchronously migrates each data table in each source database to the data table with the corresponding table structure in Analytic Database.
[0141] It should be noted here that the above-mentioned device provided by the embodiment of the present invention can implement all the method steps implemented by the above-mentioned method embodiment and can achieve the same technical effect. The parts and beneficial effects that are the same as the method embodiment in this embodiment will not be described in detail here.
[0142] Figure 5 An example of a physical structure diagram of an electronic device is shown below. Figure 5 As shown, the electronic device may include: a processor 510, a communications interface 520, a memory 530, and a communications bus 540, wherein the processor 510, the communications interface 520, and the memory 530 communicate with each other via the communications bus 540. The processor 510 may invoke logic instructions in the memory 530 to execute a business data analysis method, which includes: obtaining data set configuration information input by a user on a front-end interface; the data set configuration information includes a data source and data column definitions, the data source supports an OLAP database, including a newly added first data set, and the data column definitions support filtering conditions and analysis functions; generating a query SQL based on the data column definitions, and querying a back-end analytical database based on the query SQL to determine target business data corresponding to the first data set and data analysis results corresponding to the target business data; the analytical database stores all business data; and generating a display interface on the front-end interface; the display interface includes a data display area and / or a query user interface (UI), wherein the data display area displays the data analysis results.
[0143] Furthermore, the logic instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.
[0144] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the business data analysis method provided by the above methods, which includes: obtaining data set configuration information input by the user on the front-end interface; the above-mentioned data set configuration information includes the definition of the data source and the data column, the data source supports the OLAP database, including a newly added first data set, and the definition of the data column supports filtering conditions and analysis functions; generating a query SQL according to the definition of the data column, and querying the back-end analytical database according to the query SQL to determine the target business data corresponding to the first data set and the data analysis results corresponding to the target business data; all business data are stored in the above-mentioned analytical database; generating a display interface on the front-end interface; the above-mentioned display interface includes a data display area and / or a query user interface UI, and the data display area displays the data analysis results.
[0145] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the business data analysis method provided by the above-mentioned methods, the method comprising: obtaining data set configuration information input by a user on a front-end interface; the above-mentioned data set configuration information includes the definition of a data source and a data column, the data source supports an OLAP database, including a newly added first data set, and the definition of a data column supports filtering conditions and analysis functions; generating a query SQL according to the definition of the data column, and performing a query in a back-end analytical database according to the query SQL to determine the target business data corresponding to the first data set and the data analysis results corresponding to the target business data; the above-mentioned analytical database stores all business data; generating a display interface on the front-end interface; the above-mentioned display interface includes a data display area and / or a query user interface UI, and the data display area displays the data analysis results.
[0146] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.
[0147] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.
[0148] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.< / password> < / password>
Claims
1. A business data analysis method, characterized in that: include: Obtain the data set configuration information entered by the user on the front-end interface; The data set configuration information includes the definition of a data source and a data column. The data source supports an online analytical processing (OLAP) database, including a newly added first data set. The data column definition supports filtering conditions and analysis functions. The analysis function refers to a function that analyzes a column in a data table in an analytical database. Generate a query structured query language SQL based on the definition of the data column, and perform query and data analysis in a back-end analytical database based on the query SQL to determine the target business data corresponding to the first data set and the data analysis results corresponding to the target business data; the analytical database stores all business data; the analytical database is a StarRocks database; Generate a display interface on the front-end interface; the display interface includes a data display area and / or a query user interface UI, and the data display area displays the data analysis results; Obtaining deletion information entered by the user on the front-end interface; The deletion information includes a second data set selected by the user from the data set list on the front-end interface and a deletion operation to be performed on the second data set, wherein the data set list includes at least one data set; detecting whether the second data set has been used; The "having been used" includes that the second data set has generated an analysis list or an analysis chart, or the second data set is generating an analysis list or an analysis chart; If the second data set has been used, outputting a first prompt message; The first prompt information is used to indicate that the second data set cannot be deleted and the reason why the second data set cannot be deleted; If the second data set is not used, deleting the second data set from the data set list and executing the materialized view deletion SQL; The query and data analysis are performed in the back-end analytical database according to the query SQL to determine the target business data corresponding to the first data set and the data analysis results corresponding to the target business data, including: Performing a query in a backend analytical database according to the query SQL to determine target business data corresponding to the first data set; Generate a materialized view creation SQL based on the target business data, and control the analytical database to execute the materialized view creation SQL to generate a target materialized view corresponding to the target business data; Performing a selection operation on the target materialized view to determine a data analysis result corresponding to the target business data; The data set configuration information also includes a materialization type. Generating a materialized view creation SQL based on the target business data includes: According to the target business data and the materialization type, a materialized view creation SQL matching the materialization type is generated; wherein different materialization types correspond to different materialized view creation SQLs.
2. The business data analysis method according to claim 1, characterized in that: The data set configuration information also includes a materialization type, and the materialization type includes synchronous materialization or asynchronous materialization. The method further includes: If the materialization type is asynchronous materialization, obtaining the refresh interval and bucket fields entered by the user on the front-end interface; The updated data corresponding to the bucketed field in the first data set is periodically queried in the analytical database according to the refresh interval, and the target materialized view is updated according to the updated data.
3. The business data analysis method according to claim 1, characterized in that: The method further comprises: Receive a chart setting operation input by a user, and in response to the chart setting operation, display a chart setting interface; the chart setting interface includes a first tab interface and a second tab interface, the first tab interface is a data setting interface required for creating a chart, and the second tab interface is a style setting interface required for creating a chart; Acquiring data setting information entered by the user on the first tab interface and style setting information entered on the second tab interface; the data setting information includes at least one of a third data set selected in the data set list, selected fields for the X-axis and the Y-axis, and set specific parameters of the chart; and the style setting information includes attribute information of the chart; An analysis chart corresponding to the third data set is generated according to the data setting information and the style setting information.
4. The business data analysis method according to claim 1, characterized in that: The method further comprises: Execute data migration commands and use the data integration tool Flink CDC 3.0 to obtain data tables from different source databases in real time; A data table having the same table structure as the data table in each of the source databases is established in the analytical database, and each data table in each of the source databases is synchronously migrated to a data table in the corresponding table structure in the analytical database.
5. A business data analysis device, characterized in that: include: Configuration information acquisition module, used to obtain the data set configuration information entered by the user on the front-end interface; The data set configuration information includes the definition of a data source and a data column. The data source supports an online analytical processing (OLAP) database, including a newly added first data set. The data column definition supports filtering conditions and analysis functions. The analysis function refers to a function that analyzes a column in a data table in an analytical database. A query module is configured to generate a query structured query language (SQL) based on the definition of the data column, and perform query and data analysis in a back-end analytical database based on the query SQL to determine the target business data corresponding to the first data set and the data analysis results corresponding to the target business data; the analytical database stores all business data; the analytical database is a StarRocks database; An analysis result display module is used to generate a display interface on the front-end interface; the display interface includes a data display area and / or a user query UI area, and the data display area displays the data analysis results; A deletion information acquisition module is used to acquire the deletion information entered by the user on the front-end interface; The deletion information includes a second data set selected by the user from the data set list on the front-end interface and a deletion operation to be performed on the second data set, wherein the data set list includes at least one data set; a detection module, configured to detect whether the second data set has been used; The "having been used" includes that the second data set has generated an analysis list or an analysis chart, or the second data set is generating an analysis list or an analysis chart; an output module, configured to output first prompt information if the second data set has been used; The first prompt information is used to indicate that the second data set cannot be deleted and the reason why the second data set cannot be deleted; a deletion module, configured to delete the second data set from the data set list and execute a materialized view deletion SQL if the second data set is not used; The query module is specifically configured to query the backend analytical database according to the query SQL to determine the target business data corresponding to the first data set; generate a materialized view creation SQL according to the target business data, and control the analytical database to execute the materialized view creation SQL to generate a target materialized view corresponding to the target business data; Performing a selection operation on the target materialized view to determine a data analysis result corresponding to the target business data; The data set configuration information also includes a materialization type. The query module is specifically configured to generate a materialized view creation SQL that matches the materialization type based on the target business data and the materialization type. Different materialization types correspond to different materialized view creation SQLs.
6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the business data analysis method according to any one of claims 1 to 4 is implemented.
7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the business data analysis method according to any one of claims 1 to 4 is implemented.
Citation Information
Patent Citations
Method and system for multi-dimensional analysis of message service data
CN101197876A
OLAP multi-dimensional analysis and data mining system
CN107704608A