Pandas expression generation system based on visual interaction

Abstract

The invention discloses a Pandas expression generation system based on visual interaction, and belongs to the technical field of data processing. The front-end interaction layer is used for providing a graphical interface; the logic processing layer is used for converting a Pandas expression character string; and the expression storage layer is used for storing the generated expression in a JSON format. According to the method, a user can intuitively configure a data set, a screening condition, a grouping dimension and a calculation type through a graphical interface, and a system automatically converts configuration into a Pandas expression and stores the Pandas expression; according to the scheme, the problem that a traditional data processing tool depends on the programming capability is solved, the efficiency and autonomy of data processing are improved, and the method is suitable for scenes such as data analysis and business reports.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a Pandas expression generation system based on visual interaction. Background Technology

[0002] Pandas is a core library in Python for data processing. Its expressions (such as data filtering, grouping, and aggregation) must be implemented through programming syntax; for example, data grouping and summation can be performed using expressions like `df.groupby('category')['value'].sum()`. This presents a high learning curve for non-technical users.

[0003] Many non-technical users are accustomed to operating through graphical interfaces when processing data. Traditional data processing tools lack flexible visual configuration methods, making it difficult for users to define complex data processing logic independently, requiring assistance from technical personnel, and resulting in low efficiency. Pandas provides powerful data processing capabilities, but non-programming users cannot directly manipulate Pandas expressions, limiting their autonomy in data processing; currently, there are some auxiliary tools, but their visual configuration functions are fragmented and lack a complete expression generation and storage mechanism.

[0004] Design a method to lower the barrier to entry and enable graphical manipulation of Pandas, making it easier for various non-technical personnel to perform data processing tasks. Summary of the Invention

[0005] The purpose of this invention is to propose a Pandas expression generation system based on visual interaction; to realize the generation and storage of data processing expressions through a graphical interactive interface.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: A Pandas expression generation system based on visual interaction, including: Front-end interaction layer: provides a graphical interface; Logic processing layer: Converts Pandas expression strings; Expression storage layer: Stores the generated expressions in JSON format.

[0007] In some embodiments, the front-end interaction layer includes the following modules: dataset selection component, condition configuration panel, grouping setting area, calculation type selection, and result preview area; The dataset selection component supports importing local files or database tables; The results preview area displays the expression execution effect in real time.

[0008] In some embodiments, the condition configuration panel is configured with: fields, relationships, values, and relationships between conditions; the grouping settings area is configured with: grouping selection, sorting field selection, ascending / descending order selection; first / last selection and input; the result preview area is configured with: result type selection; and line fill input.

[0009] In some embodiments, the logic processing layer converts the user configuration into a Pandas expression string; executes the expression and returns the data result; and verifies the validity of the expression.

[0010] In some embodiments, it further includes: a real-time two-way feedback mechanism; When users perform visual configuration on the front end, the logic processing layer converts the configuration into Pandas expressions in real time and displays the data processing effect in the result preview area. If there are logical errors or field reference problems in the configuration, the user is prompted to make corrections, forming a closed-loop interactive process of "configuration-generation-preview-correction".

[0011] In some embodiments, it further includes: a scalable computing type architecture; Users can add new computational logic by implementing interface methods.

[0012] In some embodiments, the expression storage layer stores the generated expressions in JSON format, including metadata; it supports expression classification management, retrieval, and batch retrieval.

[0013] In some embodiments, the core interaction flow during use is as follows: Users import datasets, the system parses the field structure and displays it; Configure conditions, groupings, and calculation types using visual components; The system generates Pandas expressions in real time and previews the results; Users can save expressions to the backend, supporting naming and tagging; New data can then be processed by directly retrieving and calling the stored expressions.

[0014] In some embodiments, the front-end interaction uses HTML / CSS / JavaScript to develop the interface, and combines Vue.js to implement component-based interaction; the back-end logic uses the Python Flask framework to build the service.

[0015] In some embodiments, the functional implementation logic is as follows: ① Variable definition loading Check if a variable definition list exists in the configuration data; if not, skip the entire variable calculation process. Process the variable definitions in the configuration one by one. Each variable contains a variable name and a calculated expression. ② Variable calculation state control By checking whether the variable name already exists, duplicate calculations are avoided; Status flag: Update the variable status to "1: Start calculation" to track execution progress; ③ Dynamic code execution Concatenate the variable name and expression into an assignment statement; store the result. ④ Result Processing and Formatting If the result is a floating-point number, it should be formatted. ⑤ Exception handling It catches expression syntax errors and throws an error message with the variable name; Capture all other exceptions, throw custom error messages, and standardize the error message format; ⑥ Status Update The status is updated after the calculation is completed, and used for subsequent process judgment.

[0016] Compared with existing technologies, the present invention provides a Pandas expression generation system based on visual interaction, which has the following beneficial effects.

[0017] 1. This invention allows users to intuitively configure datasets, filtering conditions, grouping dimensions, and calculation types through a graphical interface. The system automatically converts the configuration into Pandas expressions and stores them. This solution solves the problem of traditional data processing tools' dependence on programming capabilities, improves the efficiency and autonomy of data processing, and is suitable for scenarios such as data analysis and business reports.

[0018] Other advantages, objectives and features of the invention will be set forth in part in the description which follows; and in part will be apparent to those skilled in the art upon examination of the following description; or may be learned from practice of the invention. Attached Figure Description

[0019] Figure 1 This is a logic diagram for implementing the present invention.

[0020] Figure 2 This is a diagram of the front-end interface of the present invention. Detailed Implementation

[0021] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0022] This Pandas expression generation system, based on visual interaction, allows users to perform data filtering, grouping, calculation, and other operations without writing code, and store the expressions for reuse.

[0023] The system architecture of this invention comprises a three-layer structure: a front-end interaction layer, a logic processing layer, and an expression storage layer. The front-end interaction layer displays the visual configuration interface and collects user input; the logic processing layer converts the configuration into Pandas expressions and performs a preview; the expression storage layer stores the expressions and configuration parameters, supporting retrieval and retrieval.

[0024] Front-end interaction layer: Provides a graphical interface and includes the following modules: Dataset selection component: Supports importing local files (CSV / Excel) or database tables; Condition configuration panel: Configure filter conditions (such as column>100) through drop-down menus, input boxes, etc. Group settings area: Supports multi-dimensional group selection (such as grouping by category and date); Calculation type selection: Includes detail types (such as returning raw data) and summary types (sum, average, count, etc.); Results preview area: Displays the expression execution effect in real time.

[0025] Logic processing layer: Convert user configuration to Pandas expression strings (e.g., convert grouped summation configuration to groupby().agg() syntax). Execute the expression and return the data result; Validate the validity of expressions (e.g., field existence, calculation logic conflicts).

[0026] Expression storage layer: The generated expression is stored in JSON format, including metadata (name, creation time, parameter configuration). It supports expression categorization, retrieval, and batch calling.

[0027] In some embodiments, it further includes: a real-time two-way feedback mechanism; When users perform visual configuration on the front end, the logic processing layer converts the configuration into Pandas expressions in real time and displays the data processing effect in the result preview area. If there are logical errors or field reference problems in the configuration, the user is prompted to make corrections, forming a closed-loop interactive process of "configuration-generation-preview-correction".

[0028] In some embodiments, it also includes: an extensible computation type architecture; and users can add new computation logic by implementing interface methods.

[0029] The core interaction flow of this invention is as follows: Users import datasets, the system parses the field structure and displays it; Configure conditions, groupings, and calculation types using visual components; The system generates Pandas expressions in real time and previews the results; Users can save expressions to the backend, supporting naming and tagging; New data can then be processed by directly retrieving and calling the stored expressions.

[0030] This invention provides a graphical interface for receiving user configurations of datasets, filtering conditions, grouping dimensions, and calculation types; mapping the configurations to Pandas expression strings and storing them in the background; and supporting the invocation of stored expressions to perform data processing operations on the dataset.

[0031] In this invention: Visual mapping mechanism: The Pandas programming syntax is broken down into visual configuration items, and executable expressions are generated through parameter mapping; Expression caching and reuse: Establishes the association between expressions and configuration parameters, supporting reuse across datasets; Real-time preview feedback: The data processing effect is displayed instantly during the user configuration process, reducing the cognitive cost of operation; Scalable computing architecture: Supports custom extended computing types to adapt to diverse business scenarios.

[0032] This system uses HTML / CSS / JavaScript to develop the front-end interface, and combines Vue.js to implement component-based interaction; for example... Figure 2 The image shown is a screenshot of the front-end interface.

[0033] The condition configuration panel allows you to set: fields, relationships, values, and relationships between conditions; The grouping settings area includes: group selection, sorting field selection, ascending / descending order selection; first / last selection and input; The results preview area is equipped with: result type selection (including: details, summary, none); and inline fill input.

[0034] The backend logic of this system is built using the Python Flask framework.

[0035] Example code for core expression calculation logic: if(config_data.get("vars") is not None): for one_var in config_data["vars"]: # Only calculate variables that have not been calculated if(ds_dict.get(one_var["name"]) is not None): continue one_var['exec_stat']='1: Start calculating variables' try: exec(one_var["name"]+"="+one_var["last_statement"],ds_dict) val = ds_dict[one_var["name"]] if isinstance(val, float): val = round(val, 2) if float(val) - int(val) == 0: val = int(val) ds_dict[one_var["name"]]=val except SyntaxError as e: raise SyntaxError(f'Syntax error in variable <{one_var["name"]}> definition: '+e.text) except Exception as e: raise SyntaxError(f'Syntax error in variable <{one_var["name"]}>:'+str(e)) one_var['exec_stat']='9: Variable calculation successful' The implementation logic is as follows.

[0036] ① Variable definition loading Conditional statement: if(config_data.get("vars") is not None) Check if a variable definition list (vars field) exists in the configuration data; if not, skip the entire variable calculation process.

[0037] Iterating through variables: for one_var in config_data["vars"] Process the variable definitions in the configuration one by one. Each variable contains name (variable name) and last_statement (calculated expression).

[0038] ② Variable calculation state control Skip computed variables: if(ds_dict.get(one_var["name"]) is not None): continue This avoids duplicate calculations by checking if the variable name already exists in ds_dict (the data storage dictionary).

[0039] Status flag: one_var['exec_stat']='1: Start calculating variables' Update the variable status to "1: Start Calculation" to track the execution progress and facilitate troubleshooting.

[0040] ③ Dynamic code execution Expression concatenation: exec(one_var["name"]+"="+one_var["last_statement"],ds_dict) Here, one_var["name"] is the variable name, one_var["last_statement"] is the evaluation expression; the variable name and the expression are concatenated to form an assignment statement; after execution, the result is stored in ds_dict (data dictionary).

[0041] ④ Result Processing and Formatting Type checking: if isinstance(val, float) If the result is a floating-point number, it should be formatted. Keep two decimal places and convert to an integer (e.g., 3.0 → 3).

[0042] ⑤ Exception handling Syntax error capture: except SyntaxError as e It catches expression syntax errors and throws an error message with the variable name; Generalized exception handling: except Exception as e Catch all other exceptions, throw custom error messages, and standardize the error message format.

[0043] ⑥ Status Update Success flag: one_var['exec_stat']='9: Variable calculation successful' The status is updated after the calculation is completed, and used for subsequent process judgment.

[0044] The expression storage design of this system uses JSON text format to store the expressions in the database.

[0045] Storage logic example: { "ds": "WeChat text", "exec_stat": "9: Variable calculation successful", "group": [], "last_statement": "__tmp_ds__=WeChat Text[(WeChat Text['Organization Name']==\"Hefei\")]\nHefei Today = ('\\n'.join([str(x) for x in ( \"\" + __tmp_ds__[\"Daily Achievement\"].astype(str) +\"\" ).values]))", "line_join": "\\n", "name": "Hefei that day", "resultType": "detail", "select": [ { "$cellEdit": true, "$index": 0, "_index": "", "after": "", "before": "", "field": "Daily Achievement" } ], "sort": "False", "sort_field": "", "sum_field": "", "sum_op": "sum", "topType": "top", "var_type": "detail", "where": [ { "$cellEdit": true, "$index": 0, "_index": "", "field": "Institution Name", "op": "==", "op_op": "&", value: "\"Hefei\"" } ] } This application has at least the following innovations.

[0046] Part 1: Deep integration of visualization and programming syntax Existing data processing tools mostly rely on preset templates or simple drag-and-drop operations, which cannot generate complex Pandas expressions. This invention breaks down the core syntax elements of Pandas expressions, such as filtering conditions, grouping logic, and aggregation calculations, into visual configuration items and establishes precise parameter mapping rules.

[0047] For example, in the conditional configuration panel, users can select fields, operators (such as ">", "<"), and values, and the system can automatically convert them into Pandas filtering syntax: df[df['column']>value]. This method of transforming professional programming syntax into visual operations is groundbreaking among similar tools and greatly lowers the barrier to entry for using Pandas.

[0048] II: Expression Lifecycle Management Mechanism Currently available data visualization tools generally lack systematic storage and reuse capabilities for data processing logic. This invention proposes a full lifecycle management mechanism for expressions, which not only stores generated Pandas expressions in JSON format but also records detailed configuration metadata, including information such as user-selected datasets, filtering conditions, grouping dimensions, and calculation types. Users can quickly retrieve stored expressions to process new datasets by searching for expression names, tags, or creation times. This mechanism enables the accumulation and reuse of data processing logic, significantly improving efficiency and flexibility compared to traditional tools that can only be configured and used once.

[0049] Three: Real-time two-way feedback mechanism Existing data visualization tools often only display results after user configuration, failing to promptly identify configuration errors. This invention constructs a real-time two-way feedback mechanism. While the user performs visual configuration on the front end, the logic processing layer converts the configuration into Pandas expressions in real time and displays the data processing effect in the results preview area. If there are logical errors or field reference problems in the configuration, the system immediately prompts the user to correct them through the interface, forming a closed-loop interactive process of "configuration-generation-preview-correction." This real-time feedback mechanism improves the accuracy of user configuration and operational efficiency, clearly distinguishing it from traditional tools.

[0050] 4: Scalable Computational Architecture Most similar tools only support a limited number of fixed calculation types, making it difficult to meet diverse business needs. This invention designs a scalable calculation type architecture, allowing users to customize and extend calculation types according to actual business requirements. The system provides standard calculation type interfaces, allowing users to add new calculation logic (such as custom statistical functions or complex data transformation algorithms) by implementing the interface methods. This scalable design enables the system to adapt to the data processing needs of different industries and scenarios, exhibiting greater versatility and adaptability.

[0051] This invention discloses a Pandas expression visualization generation system for non-programming users. Through a graphical interface, users can intuitively configure datasets, filtering conditions, grouping dimensions, and calculation types. The system automatically converts the configuration into Pandas expressions and stores them. This solution solves the problem of traditional data processing tools' dependence on programming capabilities, improves the efficiency and autonomy of data processing, and is suitable for scenarios such as data analysis and business reports. This invention features innovations such as deep integration of visualization and programming syntax, full lifecycle management of expressions, real-time two-way feedback, and a scalable computing architecture, giving it significant advantages over existing technologies.

[0052] This invention lowers the barrier to entry: non-programming users can complete Pandas data processing simply by clicking and selecting, without needing to learn Python syntax; improves efficiency: expressions are reusable, avoiding redundant configuration and improving efficiency by more than 50% when processing similar data; enhances flexibility and scalability: supports custom extended calculation types to adapt to the data processing needs of different business scenarios; and improves accuracy: a real-time two-way feedback mechanism helps users promptly identify and correct configuration errors, reducing operational mistakes.

[0053] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

[0054] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0055] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A Pandas expression generation system based on visualized interaction, characterized in that, Comprise: Front-end interaction layer: provides a graphical interface; Logical processing layer: converts Pandas expression string; Expression storage layer: stores the generated expression in JSON format.

2. The visual interaction based Pandas expression generation system of claim 1, wherein, The front-end interaction layer includes the following modules: dataset selection component, condition configuration panel, grouping setting area, calculation type selection, result preview area; The dataset selection component supports local file or database table import; The result preview area shows the expression execution effect in real time.

3. The visual interaction based Pandas expression generation system of claim 2, wherein, The condition configuration panel is set with: field, relationship, value and the relationship between conditions; The grouping setting area is set with: grouping selection, sorting field selection, ascending / descending selection; The first few / last few selection and input; The result preview area is set with: result type selection; Row filling input.

4. The visual interaction based Pandas expression generation system of claim 1, wherein, The logical processing layer converts user configuration into Pandas expression string; Execute the expression and return the data result; Verify the legality of the expression.

5. The visual interaction based Pandas expression generation system of claim 1, wherein, Also include: Real-time two-way feedback mechanism; When the user configures visually on the front end, the logical processing layer converts the configuration into a Pandas expression in real time, and displays the data processing effect in the result preview area; If there is a logical error or field reference problem, prompt the user to correct it, forming a "configuration-generation-preview-correction" closed-loop interaction process.

6. The visual interaction based Pandas expression generation system of claim 1, wherein, Also include: Extensible calculation type architecture; Users can add new calculation logic by implementing interface methods.

7. The visual interaction based Pandas expression generation system of claim 1, wherein, The expression storage layer stores the generated expression in JSON format, including metadata; Support expression classification management, retrieval and batch invocation.

8. The visual interaction based Pandas expression generation system of claim 1, wherein, The core interaction process when using is as follows: The user imports the dataset, and the system parses the field structure and displays it; Configure conditions, grouping, and calculation type through visual components; The system generates a Pandas expression in real time and previews the results; The user saves the expression to the background, supports naming and label marking; Subsequently, stored expressions can be directly invoked to process new data through retrieval.

9. The visual interaction based Pandas expression generation system of claim 1, wherein, The front-end interaction uses HTML / CSS / JavaScript to develop the interface, combined with Vue.js to realize componentized interaction; The back-end logic uses the Python Flask framework to build the service.

10. The visual interaction based Pandas expression generation system of claim 1, wherein, The function implementation logic is as follows: ① Variable definition loading Check if there is a variable definition list in the configuration data; If not, skip the entire variable calculation process; Process each variable definition item in the configuration, which includes variable name and calculation expression; ② Variable calculation state control Avoid repeated calculation by checking if the variable name already exists; State marking: update the variable state to "1: start calculation"; ③ Dynamic code execution Concatenate the variable name and expression into an assignment statement; Store the result; ④ Result processing and formatting If the result is a floating-point number, perform formatting; ⑤ Exception handling Catch syntax errors in the expression and throw an error message with the variable name; Catch all other exceptions and throw a custom error message, with a unified error prompt format; ⑥ State update Update the state after calculation, which is used for subsequent process judgment.

Images