Autonomous data analysis and construction method and system for graph modeling drawing board operation

Abstract

The invention discloses an autonomous data analysis and construction method and system for a graph modeling drawing board operation. The method comprises the following steps: packaging data in a target file into a data source; performing graph modeling drawing board operation on the data source: classifying data in the data source according to a business theme to obtain a plurality of sub-data sets, and adding a business theme label to each sub-data set; converting each sub-data set with the business theme label into a data table; classifying fields in each data table according to field attributes, and adding a field classification tag to each type of fields to form a field list directory; configuring a screening rule for each type of fields, and establishing a mapping relationship between the type of fields and the associated data table; and generating a visual data analysis model based on the operation result of the graph modeling drawing board. The method is based on graph modeling drawing board operation, codes do not need to be used, inter-table association logic is directly defined, non-technical personnel can directly participate in data analysis, and the technical threshold is lowered.

Description

Technical Field

[0001] This invention relates to the field of data analysis and processing technology, and more specifically to an autonomous data analysis construction method and system based on graphical drawing board operations. Background Technology

[0002] According to Gartner, the global enterprise data volume grew by 45% in 2023, but only 35% of enterprises could effectively utilize the value of their data. The core bottleneck lies in the disconnect between analytical tools and business needs. Visual self-service analytics dashboards, with their "what you see is what you get" operation mode, directly address business problems (such as "comparing user purchasing behavior across different channels"), becoming a key tool for enterprises to achieve data democratization.

[0003] In fields such as financial risk control and IoT monitoring, real-time data correlation and anomaly detection directly impact business results. Traditional ETL tools (such as Informatica) primarily rely on offline batch processing, resulting in slow data updates and an inability to meet the demands of real-time scenarios. With the increasing need to integrate multi-source heterogeneous data (structured tables, JSON logs, time-series data), traditional single-table analysis relying on programming tools like SQL and Python for correlation processing is no longer suitable. The following technical shortcomings exist:

[0004] (1) High technical barriers, requiring professional personnel to write code, and high collaboration costs.

[0005] (2) The process is not visible, making debugging difficult.

[0006] (3) The result is static and cannot be dynamically adjusted.

[0007] Furthermore, in large organizations, data analysis processes often suffer from knowledge loss due to staff turnover. This leads to confusion and ambiguity in personnel process information, causing serious personnel and economic losses for the company and group.

[0008] Therefore, how to improve the efficiency and accuracy of data processing, reduce the complexity of data processing, and make data association operations easier to manage and optimize are problems that urgently need to be solved by those skilled in the art. Summary of the Invention

[0009] In view of the above problems, the present invention provides an autonomous data analysis and construction method and system for graphical drawing board operation, so as to at least solve some of the technical problems mentioned in the background art.

[0010] To achieve the above objectives, the present invention adopts the following technical solution:

[0011] On the one hand, this invention provides an autonomous data analysis and construction method for graphical drawing board operations, including:

[0012] S1. Package the data in the target file into a data source;

[0013] S2. Perform a drawing template conversion operation on the data source; the drawing template conversion operation includes:

[0014] S21. Classify the data in the data source according to the business theme to obtain multiple sub-data sets of business theme domains, and add corresponding business theme tags to each sub-data set;

[0015] S22. Convert each subset of data with a business topic label into a corresponding data table;

[0016] S23. Categorize the fields in each data table according to their field attributes, and add corresponding field category labels to each category of fields to form a field list directory;

[0017] S24. Configure corresponding filtering rules for each type of field and establish a mapping relationship between the field and the associated data table;

[0018] S3. Based on the results of the graphical drawing board operation, generate a visual data analysis model to support data query, analysis and display in downstream tasks.

[0019] Furthermore, step S2 also includes:

[0020] S25. Set the connection conditions between each data table to define the data association method between each data table; the connection conditions include field matching rules, connection type and connection direction.

[0021] Furthermore, on the page corresponding to the graphical drawing board operation, it is supported to drag and drop data tables and draw connection lines to set the connection conditions between data tables.

[0022] Furthermore, step S25 also includes:

[0023] Configure the connection ports for each data table. The connection ports include input ports and output ports, which are used to define the direction and format of data inflow and outflow.

[0024] Furthermore, the graphical drawing board operation also includes graphical computational interaction functions;

[0025] The graphical computational interaction function is used to parse the graphical operations on the canvas interface and generate corresponding related computational logic.

[0026] Furthermore, the filtering rules include single-condition filtering and multi-condition combination filtering.

[0027] On the other hand, the present invention provides an autonomous data analysis and construction system for graphical drawing board operation, characterized by the application of the above-mentioned method; the system includes a data source packaging module, a graphical drawing board operation module, and a data analysis model generation module;

[0028] The data source packaging module is used to package the data in the target file into a data source;

[0029] The graphic template drawing board operation module includes:

[0030] The data source classification submodule is used to classify the data in the data source according to the business theme, obtain multiple sub-data sets of business theme domains, and add corresponding business theme labels to each sub-data set;

[0031] The data table generation submodule is used to convert each subset of data with a business theme tag into a corresponding data table;

[0032] The field classification submodule is used to classify the fields in each data table according to their field attributes, and add corresponding field classification labels to each category of fields to form a field list directory;

[0033] The filter rule configuration submodule is used to configure the corresponding filter rules for each type of field and establish the mapping relationship between the field and the associated data table;

[0034] The data analysis model generation module is used to generate a visual data analysis model based on the results of the graphical drawing board operation, which is used to support data query, analysis and display in downstream tasks.

[0035] Furthermore, the graphic template operation module also includes a connection condition setting submodule;

[0036] The connection condition setting submodule is used to set the connection conditions between each data table and to define the data association method between each data table; the connection conditions include field matching rules, connection type and connection direction.

[0037] Furthermore, the graphic template operation module also includes a connection port setting submodule;

[0038] The connection port setting submodule is used to configure the connection ports of each data table. The connection ports include input ports and output ports, which are used to define the direction and format of data inflow and outflow.

[0039] Furthermore, the graphical drawing board operation module also includes a graphical computational interaction function submodule;

[0040] The graphical computational interaction function submodule is used to parse the graphical operations on the canvas interface and generate corresponding related computational logic.

[0041] As can be seen from the above technical solution, compared with the prior art, the present invention discloses an autonomous data analysis and construction method and system for graphical drawing board operation, which has the following beneficial effects:

[0042] This invention is based on graphical drawing board operation, which allows users to define the relationship logic between tables directly through graphical operations such as selection and filtering without using code. This allows non-technical personnel, especially grassroots business personnel in enterprises, to directly participate in data analysis, thus lowering the technical threshold.

[0043] The graphical modeling method in this invention significantly improves data processing efficiency through a graphical interface and automated calculation process. It reduces the time and error rate associated with manually writing SQL statements or data processing logic.

[0044] The graph-based operation method in this invention supports various data table association operations, allowing users to flexibly select and combine different operation methods according to actual needs. It also supports dynamic adjustment and optimization of data association relationships to adapt to different data processing scenarios.

[0045] The graphical interface in this invention forces explicit definition of association logic, thus avoiding hidden errors in the code.

[0046] This invention can support complex scenarios and handle multi-table joint analysis needs (such as user-order-product three-level association) through nested and multi-level association operations, and has strong scalability.

[0047] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0049] Figure 1 This is a schematic diagram of the autonomous data analysis and construction method for graphical drawing board operation provided in an embodiment of the present invention.

[0050] Figure 2 This is a schematic diagram of the drawing board operation provided in an embodiment of the present invention.

[0051] Figure 3This is a schematic diagram of the internal logic of the graphical drawing board operation provided in an embodiment of the present invention.

[0052] Figure 4 This is a schematic diagram illustrating a power supply example provided in an embodiment of the present invention. Detailed Implementation

[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] Example 1:

[0055] This invention discloses an autonomous data analysis and construction method for graphical drawing board operations. See [link to relevant documentation]. Figure 1 As shown, it includes:

[0056] S1. Package the data in the target file into a data source;

[0057] S2. Perform a drawing template conversion on the data source; the drawing template conversion includes:

[0058] S21. Classify the data in the data source according to the business theme to obtain multiple business theme domains of sub-datasets, and add corresponding business theme labels to each sub-dataset.

[0059] S22. Convert each subset of data with a business topic label into a corresponding data table;

[0060] S23. Categorize the fields in each data table according to their field attributes, and add corresponding field category labels to each category of fields to form a field list directory;

[0061] S24. Configure corresponding filtering rules for each type of field and establish a mapping relationship between the field and the associated data table;

[0062] S3. Based on the results of the graphical drawing board operation, generate a visual data analysis model to support data query, analysis and display in downstream tasks.

[0063] Next, each of the above steps will be explained in detail.

[0064] In step S1 above, the data in the target file is packaged into a data source; the target file includes files such as Excel and txt.

[0065] For example, a data source for "modern power supply service analysis" can be constructed based on the user electricity information in target files within a certain community, including electricity consumption records analysis, electricity settlement analysis, and electricity work order analysis.

[0066] In step S2 above, a drawing board operation is performed on the data source; participate Figure 2 As shown, the drawing template conversion process specifically includes the following steps:

[0067] S21. Classify the data in the data source according to the business theme to obtain multiple business theme domains of sub-datasets, and add corresponding business theme labels to each sub-dataset.

[0068] For example, by using "electricity and electricity charges receivable" as the classification criterion, the above-mentioned "modern power supply service analysis" data sources can be classified to obtain multiple sub-datasets related to "modern power supply service analysis";

[0069] S22. Transform each subset of data with a business theme tag into a corresponding data table; the generated data table contains all the information of the corresponding data source, and by consulting the data table, the overall data of the data source can be understood in different categories;

[0070] For example, the above-mentioned multiple subsets of data related to "modern power supply service analysis" can be converted into corresponding data tables;

[0071] S23. The data table contains more detailed classification information, called fields. The fields contain more detailed data information from the data source. The fields in each data table are classified according to their attributes, and corresponding field classification labels are added to each category of fields to form a field list directory.

[0072] For example, the fields in the aforementioned data tables related to "Modern Power Supply Service Analysis" can be further divided into "User Number", "User Name", "User Category", "Power Supply Unit Code", "Asset Number", "Metering Point Number", "Regional Code", "Metering Method", "Phase Line Code", "Integrated Grid Connection Rate", "PT Transformer Ratio Code", "Electricity Capacity", "Operating Capacity", "Rated Capacity", "Billing Capacity", and "Administrative Region", forming a corresponding field list directory. Through this field list directory, different table fields can be selected to perform field list analysis or export.

[0073] S24. Configure corresponding filtering rules for each type of field and establish a mapping relationship between the field and the associated data table; the filtering rules include single-condition filtering and multi-condition combined filtering, and the rules can be aggregation rules;

[0074] For example, for the "User ID" field, set the filter rule to "equal to"; for the "Metering Method" field, set the filter rule to "equal to / less than or equal to / greater than or equal to"; for the "Operating Capacity" field, set the filter rule to "equal to / less than or equal to / greater than or equal to".

[0075] S25. Set the connection conditions between each data table to define the data association method between each data table; the connection conditions include field matching rules, connection type and connection direction; and configure the connection port of each data table, including input port and output port, to define the direction and format of data inflow and outflow;

[0076] In this embodiment of the invention, on the page corresponding to the graphic template drawing board operation, it is supported to drag and drop data tables and draw connection lines to set the connection conditions between each data table.

[0077] In this embodiment of the invention, the graphical drawing board operation also includes a graphical computational interaction function, used to parse the graphical operations on the drawing board interface and generate corresponding associated computational logic, see details below. Figure 3 As shown, this association operation logic enables table-to-table association operations, abstracting set operations into graphical operations. Graphical elements (such as nodes, lines, and icons) represent data entities (such as tables and fields) and operation logic (such as selection and filtering) to perform logical operations between data tables (such as JOIN, UNION, and INTERSECT in SQL). This innovative method not only improves the efficiency and accuracy of data processing but also reduces its complexity, making data association operations easier to manage and optimize. It has broad application prospects in database management, data analysis, data warehousing, and big data processing, demonstrating significant technical advantages and practical value.

[0078] S3. Based on the results of the graphical drawing board operation, generate a visual data analysis model to support data query, analysis and display in downstream tasks;

[0079] For example, given the data analysis model already constructed, the user finds the required field label (e.g., "Dosage Method" or "Operating Capacity") from the target business theme. Double-clicking the field label redirects to the filtering rules interface, where the user can set filtering conditions. For instance, "Dosage Method" can be set to "equal to" for "High-power high-power meter," and "Operating Capacity" can be set to "less than or equal to," "equal to," or "greater than or equal to" for "630." After setting and confirming the conditions, the system filters and displays the data from the corresponding data table. A schematic diagram of the implementation process for the power supply case above can be found here. Figure 4 As shown.

[0080] In summary, the autonomous data analysis and construction method based on a graphical drawing board provided by this invention can quickly complete the integrated operation of "finding data + viewing data + using data," achieving a WYSIWYG user experience; specifically:

[0081] By connecting to the data source and splitting the data into columns, the source data is displayed in the form of a database table, forming detailed data fields, thus completing the task of "finding data".

[0082] For data that has completed data source access and data classification, the system automatically performs detailed data classification and selects data directories. Users can then double-click on a data field to select that field and use it as one of the filtering conditions. The system automatically associates relationships between different tables and stores the fields selected by the user, initially filtering out the data the user needs and completing the "data viewing" task.

[0083] The data that has been initially screened can be further filtered by the "design range". Users can then select the "accurate range" or set conditions to complete the automatic association of relationships between tables and the setting of filtering conditions. This allows users to accurately select the data they specifically need and achieve the "data usage" task.

[0084] It's important to clarify here that the "data viewing" process involves categorizing and filtering all data, performing correlation calculations based on broad categories such as "operating capacity, metering method, rated capacity, and user category." The "data usage" process further defines the specific "design range" of the capacity within the "operating capacity" range from the "data viewing" process. For example, "less than or equal to 630" in "operating capacity" is a further data-driven and quantitative filtering based on the "category" conditions selected during the "data viewing" process. This allows customers to achieve accurate "data usage" and a WYSIWYG (What You See Is What You Get) operational experience.

[0085] The embodiments of the present invention can achieve a data operation experience of "low-code development and WYSIWYG" through data operations of "finding data + viewing data + using data".

[0086] Example 2:

[0087] This invention also provides an autonomous data analysis and construction system for graphical drawing board operations, applying the above-described method; the system includes a data source packaging module, a graphical drawing board operation module, and a data analysis model generation module; specifically:

[0088] (1) Data source packaging module, used to package data in the target file into a data source;

[0089] (2) The drawing template operation module includes:

[0090] 1) The data source classification submodule is used to classify the data in the data source according to the business theme, obtain multiple business theme domains of sub-datasets, and add corresponding business theme labels to each sub-dataset;

[0091] 2) The data table generation submodule is used to convert each subset of data with a business theme tag into a corresponding data table;

[0092] 3) Field classification submodule, used to classify fields in each data table according to field attributes, and add corresponding field classification labels to each category of fields to form a field list directory;

[0093] 4) The filter rule configuration submodule is used to configure the corresponding filter rules for each type of field and establish the mapping relationship between the field and the associated data table;

[0094] 5) The connection condition setting submodule is used to set the connection conditions between various data tables and to define the data association methods between them. The connection conditions include field matching rules, connection type, and connection direction.

[0095] 6) Connection Port Settings submodule, used to configure the connection ports of each data table. The connection ports include input ports and output ports, used to define the direction and format of data inflow and outflow;

[0096] 7) Graphical computation interaction function submodule, used to parse the graphical operations on the canvas interface and generate the corresponding related computation logic;

[0097] (3) Data analysis model generation module, which is used to generate a visual data analysis model based on the results of the graphical drawing board operation, to support data query, analysis and display in downstream tasks.

[0098] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0099] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for autonomous data analysis and construction based on graphical drawing board operations, characterized in that, include: S1. Package the data in the target file into a data source; S2. Perform a drawing board operation on the data source; The drawing template operation includes: S21. Classify the data in the data source according to the business theme to obtain multiple sub-data sets of business theme domains, and add corresponding business theme tags to each sub-data set; S22. Convert each subset of data with a business topic label into a corresponding data table; S23. Categorize the fields in each data table according to their field attributes, and add corresponding field category labels to each category of fields to form a field list directory; S24. Configure corresponding filtering rules for each type of field and establish a mapping relationship between the field and the associated data table; S3. Based on the results of the graphical drawing board operation, generate a visual data analysis model to support data query, analysis and display in downstream tasks.

2. The autonomous data analysis and construction method for graphical drawing board operation according to claim 1, characterized in that, Step S2 further includes: S25. Set the connection conditions between each data table to define the data association method between each data table; the connection conditions include field matching rules, connection type and connection direction.

3. The autonomous data analysis and construction method for graphical drawing board operation according to claim 2, characterized in that, On the page corresponding to the graphical drawing board operation, it is supported to drag and drop data tables and draw connection lines to set the connection conditions between data tables.

4. The autonomous data analysis and construction method for graphical drawing board operation according to claim 2, characterized in that, Step S25 further includes: Configure the connection ports for each data table. The connection ports include input ports and output ports, which are used to define the direction and format of data inflow and outflow.

5. The autonomous data analysis and construction method for graphical drawing board operation according to claim 1, characterized in that, The graphical drawing board operation also includes graphical computational interaction functions; The graphical computational interaction function is used to parse the graphical operations on the canvas interface and generate corresponding related computational logic.

6. The autonomous data analysis and construction method for graphical drawing board operation according to claim 1, characterized in that, The filtering rules include single-condition filtering and multi-condition combination filtering.

7. A self-contained data analysis and construction system based on a graphical drawing board, characterized in that, The system comprises a data source packaging module, a graphical drawing board operation module, and a data analysis model generation module, using any one of claims 1-6. The data source packaging module is used to package the data in the target file into a data source; The graphic template drawing board operation module includes: The data source classification submodule is used to classify the data in the data source according to the business theme, obtain multiple sub-data sets of business theme domains, and add corresponding business theme labels to each sub-data set; The data table generation submodule is used to convert each subset of data with a business theme tag into a corresponding data table; The field classification submodule is used to classify the fields in each data table according to their field attributes, and add corresponding field classification labels to each category of fields to form a field list directory; The filter rule configuration submodule is used to configure the corresponding filter rules for each type of field and establish the mapping relationship between the field and the associated data table; The data analysis model generation module is used to generate a visual data analysis model based on the results of the graphical drawing board operation, which is used to support data query, analysis and display in downstream tasks.

8. The autonomous data analysis and construction system for graphical drawing board operation according to claim 7, characterized in that, The graphic template operation module also includes a connection condition setting submodule; The connection condition setting submodule is used to set the connection conditions between each data table and to define the data association method between each data table; the connection conditions include field matching rules, connection type and connection direction.

9. The autonomous data analysis and construction system for graphical drawing board operation according to claim 7, characterized in that, The graphic template operation module also includes a connection port setting submodule; The connection port setting submodule is used to configure the connection ports of each data table. The connection ports include input ports and output ports, which are used to define the direction and format of data inflow and outflow.

10. The autonomous data analysis and construction system for graphical drawing board operation according to claim 7, characterized in that, The graphical drawing board operation module also includes a graphical calculation and interaction function sub-module; The graphical computational interaction function submodule is used to parse the graphical operations on the canvas interface and generate corresponding related computational logic.

Images