Visual big data auxiliary analysis method

Abstract

The invention discloses a visual big data auxiliary analysis method. The method comprises the following steps of 1, configuring an SQL (Structured Query Language) to obtain data; 2, configuring a tag app (tagapp); step 3, binding the application tagapp and the main body tagphy by the tag; 4, storing the data into a TD engine database in a labeling mode, and collecting the data through a scheduling task; and 5, displaying a visual result. According to the method, the user can easily define the own data model, various types of data charts and data tables are freely combined, and deep mining and personalized display of the data are realized. The system not only supports traditional structured data processing, but also can effectively integrate semi-structured data and non-structured data, and meets cross-field and cross-industry big data analysis requirements. The method can support users to randomly add various types of data graphs and data tables according to own ideas, has high flexibility and expandability, and meets the requirements of the users for flexibly constructing complex data graphs and data tables according to actual requirements.

Description

Technical Field

[0001] The present invention belongs to the technical field of electrical data processing, and in particular relates to a visualized big data auxiliary analysis method. Background Art

[0002] With the rapid development of information technology, big data has become an indispensable resource for modern enterprises and research institutions. The complexity and diversity of big data require data analysis tools to be highly flexible and scalable to meet the diverse needs of different fields and application scenarios. Traditional big data analysis systems often focus on predefined data models and fixed analysis processes, which to some extent limits the ability of users to customize according to specific business scenarios or research purposes.

[0003] In the current field of big data analysis, although there are a variety of data analysis tools and platforms available, such as Excel, Tableau, Power BI, etc., which provide rich data visualization functions and certain data analysis capabilities, they still have limitations in supporting user-defined data structures and analysis processes. For example, these tools usually require users to organize data in a predetermined format and select from preset chart templates for analysis, which makes it difficult for users to flexibly build complex data charts and data tables according to actual needs.

[0004] In addition, with the increasing variety of data types, such as the coexistence of structured data, semi-structured data, and unstructured data, as well as the increasing demand for real-time data processing, traditional big data analysis systems face challenges in processing these complex data types and ensuring data real-time performance. Therefore, it is particularly important to develop a big data analysis system that can support users to freely add various types of data graphs and data tables according to their own ideas, while also having high flexibility and scalability.

[0005] In summary, how to enable users to easily define their own data models, freely combine various types of data charts and data tables, and achieve deep data mining and personalized display. It can also support traditional structured data processing, and effectively integrate semi-structured data and unstructured data to meet the needs of cross-domain and cross-industry big data analysis. Ensuring that users can instantly obtain the latest analysis results, provide strong support for decision-making, and significantly improve the efficiency of data analysis and decision-making has become a difficult problem that technicians in the field of electrical data processing need to solve. Summary of the invention

[0006] In view of the shortcomings of the prior art, the technical problem to be solved by the present invention is to provide a visual big data assisted analysis method that enables users to easily define their own data models, freely combine various types of data charts and data tables, and realize deep mining and personalized display of data.

[0007] In order to solve the above technical problems, the technical solution adopted by the present invention is: a visual big data auxiliary analysis method, characterized in that it includes the following steps: Step 1: Configure SQL to obtain data; Step 2: Configure the tag tag_app; Step 3: Bind the tag application tag_app and the subject tag_phy; Step 4: Store the data into the TDengine database by labeling and collect data by scheduling tasks; Step 5: Visualize the results.

[0008] In the above-mentioned visualized big data assisted analysis method, in step 1, the specific process of configuring SQL to obtain data is as follows: (1) Define data requirements: retrieve the required data from the database, including specific columns, tables, or complex related data across multiple tables; (2) Write SQL queries: Write corresponding SQL query statements based on data requirements, including specifying the SELECT clause to select the required columns, the FROM clause to specify the table of data sources, and the WHERE clause to filter the results; (3) Configure database connection: Before executing SQL query, pre-configure the connection with the database, which includes specifying the database server address, port, database name, user name and password and other authentication information; (4) Execute query: Execute the written SQL query statement through database connection; (5) Process query results: After executing the query, a result set is returned, which contains the data rows that meet the query conditions.

[0009] The above-mentioned visualized big data assisted analysis method is characterized in that: in the step 2, the specific process of configuring the tag tag_app is as follows: (1) Define tags: clarify the meaning and purpose of tag_app and ensure that it maintains consistency and interoperability with other parts of the system; (2) Create a tag: Create a tag_app tag in the system's tag registration and specify the tag attributes, including name, type, system attribute, unit, period, and physical classification.

[0010] The above-mentioned visualized big data assisted analysis method is characterized in that, in the step 3, the specific process of tag binding application tag_app and subject tag_phy is as follows: (1) Determine the binding relationship: clarify the association between tag_app and tag_phy; (2) Perform binding operations: Perform the actual binding operation through the tag binding page to associate specific tag_app and tag_phy values ​​with the metadata of the data item or data source.

[0011] The above-mentioned visualized big data assisted analysis method is characterized in that, in the step 4, the data is stored in the TDengine database by labeling, and the data is collected by scheduling tasks, and the specific process is as follows: (1) Configure scheduling tasks: set up task flows to perform data collection regularly; (2) Store data in TDengine: The collected data is stored in the database through TDengine's API. During the storage process, the data labels are correctly set and associated.

[0012] The above-mentioned visualized big data assisted analysis method is characterized in that, in the step 5, the specific process of displaying the visualized results is as follows: (1) Enter your account and password to log into the system; (2) Configure SQL to obtain data: Obtain data by writing SQL to associate with the database table. The SQL writing includes the Chinese and English names of the SQL, the SQL description, the warehouse type, the target warehouse, and the overall SQL statement; (3) Tag registration: fill in the tag name, attributes, type and other parameters to create the tag; (4) Tag binding: Bind the physical tag and the application tag to each other to determine the association relationship; (5) Scheduling task collection: In the scheduling task, the task scheduling is performed through the design process to collect data and store it in the database; (6) Configure visualization result charts: Select from the component library, configure the data page according to user needs, customize page preview, deletion, saving, and sharing, and modify the chart to a custom style.

[0013] The above-mentioned visualized big data assisted analysis method is characterized in that the component library for visual display includes: a table component for displaying structured data, a text control for adding titles, explanatory text or comments to enhance the readability and comprehensibility of charts, a time control for selecting or entering a time range to dynamically display the trend of data changes over time, a button that provides user interaction functions, and a mixed chart control that combines multiple chart types to display multiple dimensions and relationships of data.

[0014] The above-mentioned visualized big data assisted analysis method is characterized in that: the visualization results include curve graphs, pie charts, radar charts, dashboards, and structure trees.

[0015] The advantages of the visualized big data assisted analysis method of the present invention are: a new assisted big data analysis system is proposed, which aims to enable users to easily define their own data models and freely combine various types of data charts and data tables by providing a set of user-friendly interfaces and powerful back-end support, so as to realize deep mining and personalized display of data. The system not only supports traditional structured data processing, but also effectively integrates semi-structured data and unstructured data to meet the needs of cross-domain and cross-industry big data analysis. At the same time, by introducing advanced real-time data processing technology, it ensures that users can obtain the latest analysis results in real time, providing strong support for decision-making. It can support users to add various types of data graphs and data tables at will according to their own ideas, and at the same time has high flexibility and scalability, which meets the needs of users to flexibly build complex data charts and data tables according to actual needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A schematic diagram of the process flow of the visualized big data assisted analysis method of the present invention; Figure 2 Schematic diagram of the process for visualizing the results. DETAILED DESCRIPTION

[0017] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0018] In the present invention, unless otherwise stated, directional words such as "upper" and "lower" generally refer to the upper and lower parts of the device in actual use or working state, specifically the drawing direction in the attached drawings; while "inside" and "outside" refer to the outline of the device. In addition, in the description of this application, the term "including" means "including but not limited to". The terms first, second, third, etc. are used only as labels and do not impose numerical requirements or establish an order. The term "plurality" means "two or more".

[0019] like Figure 1 , 2As shown, a visual big data assisted analysis method includes the following steps: Step 1: Configure SQL to obtain data; Step 2: Configure the tag tag_app; Step 3: Bind the tag application tag_app and the subject tag_phy; Step 4: Store the data into the TDengine database by labeling and collect data by scheduling tasks; Step 5: Visualize the results.

[0020] Among them, the specific process of step 1, configuring SQL to obtain data is as follows: (1) Define data requirements: First, clarify what data needs to be retrieved from the database. This may include specific columns, tables, or even complex related data across multiple tables.

[0021] (2) Write SQL queries: Write the corresponding SQL query statements according to data requirements. This includes specifying the SELECT clause to select the required columns, the FROM clause to specify the table where the data comes from, and possible WHERE clauses to filter the results. For complex queries, it may also involve JOIN clauses to link multiple tables, GROUP BY clauses to group data, and ORDER BY clauses to sort the results.

[0022] (3) Configure the database connection: Before you can execute SQL queries, you need to configure the connection to the database. This usually involves specifying authentication information such as the database server address, port, database name, username and password. In a programming environment, this can be achieved through a database connection string or a connection object.

[0023] (4) Execute query: Execute the written SQL query statement through the database connection. This usually involves calling a specific function or method in the database API and passing in the SQL statement as a parameter.

[0024] (5) Processing query results: After executing the query, a result set will be returned, which contains the data rows that meet the query conditions. Depending on the needs of the application, these results need to be further processed, such as converted into object lists, mapped to data models, statistically calculated, or directly displayed to users.

[0025] In step 2, the specific process of configuring the tag tag_app is as follows: (1) Define tags: Clarify the meaning and purpose of tag_app and ensure it remains consistent and interoperable with other parts of the system.

[0026] (2) Create a tag: Create a tag_app tag in the system's tag registry. Specify the tag's name, type (such as string, integer, etc.), system attribute, unit, period, and physical classification.

[0027] In step 3, the specific process of tag binding application tag_app and subject tag_phy is as follows: (1) Determine the binding relationship: clarify the association between tag_app and tag_phy. For example, an application (tag_app) may be monitoring or controlling a specific set of physical devices (tag_phy).

[0028] (2) Perform the binding operation: Perform the actual binding operation through the tag binding page. Associate specific tag_app and tag_phy values ​​with the metadata of the data item or data source.

[0029] In step 4, the data is stored in the TDengine database by labeling, and the data is collected by scheduling tasks. The specific process is as follows: (1) Configure scheduling tasks: set up task flows to perform data collection regularly; (2) Store data in TDengine: The collected data is stored in the database through TDengine's API. During the storage process, the data tags are correctly set and associated.

[0030] In step 5, the visualization results are displayed, including the following components and result display methods: 1) Table component: used to display structured data, such as table information in a database. Users can quickly browse and filter data through this component.

[0031] 2) Text control: used to add titles, explanatory text or comments to enhance the readability and comprehension of the chart.

[0032] 3) Time control: used to select or enter a time range to dynamically display the changing trend of data over time.

[0033] 4) Buttons: provide user interaction functions, such as switching chart types, refreshing data, or exporting charts.

[0034] 5) Mixed chart control: Combines multiple chart types (such as line charts and pie charts) to display multiple dimensions and relationships of data.

[0035] 6) Curve chart: used to show the trend of data changes over time, suitable for visualization of time series data.

[0036] 7) Pie chart: used to display the proportional relationship of data, so that users can quickly understand the distribution of data.

[0037] 8) Radar chart: used to display multi-dimensional comparison of data, suitable for evaluating the comprehensive performance of multiple indicators.

[0038] 9) Dashboard: simulates the traditional dashboard style and is used to display the current status of key indicators (KPIs), such as speed, temperature, etc.

[0039] 10) Structure tree: used to display tree structures, such as directories, etc.

[0040] The display method of the visualization result of the present invention is as follows: 1) Log in to the system Enter your account and password to log into the system.

[0041] 2) Configure SQL to obtain data Get data by writing SQL to associate with database tables. SQL writing includes the Chinese and English names of SQL, SQL description, warehouse type, target warehouse and the overall SQL statement.

[0042] 3) Tag registration Fill in the tag name, attributes, type and other parameters to create the tag 4) Tag binding The physical tags and application tags are bound to each other to determine the association relationship.

[0043] 5) Scheduling task collection During the task scheduling, the task scheduling is performed through the design process to collect data and store it in the database.

[0044] 6) Configure visualization chart Select from the component library on the left and configure the data page according to the customer's personal wishes. You can preview, delete, save, and share the page to customize it, and you can also modify the chart in a custom style on the right.

[0045] At the same time, for the saved charts, users can view or modify them according to the historical templates, and after the big data is connected, users can select charts for comparison, which can intuitively display the changes and quantification of data, and play an auxiliary role in big data analysis. The visualization system of the present invention integrates a variety of controls to achieve comprehensive display and flexible interaction of various types of data. Compared with the prior art, the system has the advantages of comprehensive functions, simple operation, and intuitive display effects, and can significantly improve the efficiency of data analysis and decision-making.

[0046] Of course, the above description is not a limitation of the present invention, and the present invention is not limited to the above examples. Any changes, modifications, additions or substitutions made by ordinary technicians in this technical field within the essential scope of the present invention should fall within the protection scope of the present invention.

Claims

1. A visual big data assisted analysis method, characterized in that: The steps include: Step 1: Configure SQL to obtain data; Step 2: Configure the tag tag_app; Step 3: Bind the tag application tag_app and the subject tag_phy; Step 4: Store the data into the TDengine database by labeling and collect data by scheduling tasks; Step 5: Visualize the results.

2. The visualized big data assisted analysis method according to claim 1 is characterized in that: In step 1, the specific process of configuring SQL to obtain data is as follows: (1) Define data requirements: retrieve the required data from the database, which may include columns, tables, or complex related data across multiple tables; (2) Write SQL queries: Write corresponding SQL query statements based on data requirements, including specifying the SELECT clause to select the required columns, the FROM clause to specify the table of data sources, and the WHERE clause to filter the results; (3) Configure database connection: Before executing SQL query, pre-configure the connection with the database, which includes specifying the database server address, port, database name, user name and password; (4) Execute query: Execute the written SQL query statement through database connection; (5) Process query results: After executing the query, a result set is returned, which contains the data rows that meet the query conditions.

3. The visualized big data assisted analysis method according to claim 1 is characterized in that: In step 2, the specific process of configuring the tag tag_app is as follows: (1) Define tags: clarify the meaning and purpose of tag_app and ensure that it maintains consistency and interoperability with other parts of the system; (2) Create a tag: Create a tag_app tag in the system's tag registration and specify the tag attributes, including name, type, system attribute, unit, period, and physical classification.

4. The visualized big data assisted analysis method according to claim 1 is characterized in that: In step 3, the specific process of tag binding application tag_app and subject tag_phy is as follows: (1) Determine the binding relationship: clarify the association between tag_app and tag_phy; (2) Perform the binding operation: Perform the actual binding operation through the tag binding page to associate the tag_app and tag_phy values ​​with the metadata of the data item or data source.

5. The visualized big data assisted analysis method according to claim 1 is characterized in that: In step 4, the data is stored in the TDengine database by labeling, and the data is collected by scheduling tasks. The specific process is as follows: (1) Configure scheduling tasks: set up task flows to perform data collection regularly; (2) Store data in TDengine: The collected data is stored in the database through TDengine's API. During the storage process, the data labels are correctly set and associated.

6. The visualized big data assisted analysis method according to claim 1 is characterized in that: In step 5, the specific process of displaying the visualization results is as follows: (1) Enter your account and password to log into the system; (2) Configure SQL to obtain data: Obtain data by writing SQL to associate with the database table. The SQL writing includes the Chinese and English names of the SQL, the SQL description, the warehouse type, the target warehouse, and the overall SQL statement; (3) Tag registration: fill in the tag name, attributes, and type to create the tag; (4) Tag binding: Bind the physical tag and the application tag to each other to determine the association relationship; (5) Scheduling task collection: In the scheduling task, the task scheduling is performed through the design process to collect data and store it in the database; (6) Configure visualization result charts: Select from the component library, configure the data page according to user needs, customize page preview, deletion, saving, and sharing, and modify the chart to a custom style.

7. The visualized big data assisted analysis method according to claim 6 is characterized in that: The component library for visual display includes: table components for displaying structured data, text controls for adding titles, explanatory text or comments to enhance the readability and comprehension of charts, time controls for selecting or entering time ranges to dynamically display the changing trends of data over time, buttons that provide user interaction functions, and mixed chart controls that combine multiple chart types to display multiple dimensions and relationships of data.

8. The visualized big data assisted analysis method according to claim 1 is characterized in that: The visualization results include curve charts, pie charts, radar charts, dashboards, and structure trees.

Images