Visual data processing method and device, electronic equipment, storage medium and program product

Abstract

The invention provides a visual data processing method and device, electronic equipment, a storage medium and a program product, relates to the technical field of visual data processing, and is used for realizing visual experience of plug and play of data through automatic data loading and traceless code injection. The method comprises the following steps: acquiring a data file, and storing the data file in a temporary directory; identifying the format of the data file, calling a corresponding loading function based on the format of the data file, and generating a data loading code based on the temporary directory stored in the data file and the corresponding loading function; receiving a user code from a client, and preposing the data loading code into the user code to generate a complete executable code; and executing the complete executable code, generating a data visualization result, and sending the data visualization result to the client.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of visual data processing, and in particular to a visual data processing method and device, electronic equipment, storage medium and program product. BACKGROUND

[0002] With the rapid development of information technology, data is being generated at an unprecedented speed and scale, and is penetrating into various fields such as scientific research, medical treatment, finance and industry. In the face of massive, high-dimensional and heterogeneous complex data, traditional data processing methods are often limited to numerical calculation and textual description, and it is difficult to intuitively present the internal correlation, change trend and hidden pattern of the data, resulting in low efficiency of data value mining and lack of intuitive basis in the decision-making process. As an important bridge connecting data and human cognition, visualization technology can effectively reduce the threshold of data understanding and help users quickly capture key information by converting abstract data into visual forms such as graphics and images. With the deep integration of data science and visualization technology, data visualization processing has become a core means in the fields of scientific analysis, business decision-making and engineering development.

[0003] However, in the prior art, programmatic tools cannot automatically fill in data loading code, data loading is tedious and repetitive, the operation cost is high, and it constitutes a significant usage barrier for non-technical users. Low-code tools cannot implement complex custom logic. SUMMARY

[0004] The present application provides a visual data processing method, device, electronic equipment, storage medium and program product, which realizes the "plug and play" visual experience of data through automatic data loading, traceless code injection, intelligent information extraction and secure execution environment, which can reduce the operation cost, improve the development efficiency, realize the complex data loading logic, and reduce the security risk in the open code execution scene.

[0005] In a first aspect, the present application provides a visual data processing method, which can include: obtaining a data file and storing the data file in a temporary directory; identifying the format of the data file, and calling a corresponding loading function based on the format of the data file; generating data loading code based on the temporary directory where the data file is stored and the corresponding loading function; receiving user code from a client, and prepositioning the data loading code into the user code to generate complete executable code; executing the complete executable code to generate a data visualization result, and sending the data visualization result to the client.

[0006] The above technical solutions provided by the present application at least bring the following beneficial effects: the user only needs to upload a data file, the system can automatically generate data loading code, automatically inject standardized data loading logic before the user code is executed, execute the executable code to generate data visualization results, and does not need to care about data loading details, solves the problem of complicated data loading, can realize complex data loading logic while reducing operation cost and improving development efficiency, and realizes seamless experience of "plug and play".

[0007] A possible implementation manner, which generates data loading code based on the temporary directory stored by the data file and the corresponding loading function, can include: securely escaping the file path of the data file in the temporary directory, and generating the data loading code based on the securely escaped file path and the corresponding loading function. By doing so, only the temporary directory can be accessed when generating the data loading code, to prevent path injection attacks (such as.. / etc / passwd) and malicious file reading.

[0008] A possible implementation manner, which generates data loading code based on the temporary directory stored by the data file and the corresponding loading function, can include: securely escaping the file path of the data file in the temporary directory, and generating the data loading code based on the securely escaped file path and the corresponding loading function. By doing so, only the temporary directory can be accessed when generating the data loading code, to prevent path injection attacks (such as.. / etc / passwd) and malicious file reading.

[0009] Another possible implementation manner, which generates global data objects and / or metadata information for the user code based on the data file, can include: reading data in the data file using the corresponding loading function; converting the data in the data file into predetermined format data to generate the global data objects; and / or extracting the metadata information from the data in the data file.

[0010] Another possible implementation manner, the metadata information can include column names, data types, and data shapes.

[0011] Another possible implementation manner, the metadata information can be used to provide code hints for the user to help the user complete the user code when the user edits the user code. By doing so, completion suggestions can be generated in real time when the user writes the user code (for example, all column name options are automatically popped up when df[' is input), reducing syntax errors and further improving development efficiency.

[0012] In another possible implementation, the global data object and the metadata information can be used to generate an initial template code of the user code. In this way, the initial template code can be automatically generated by algorithm analysis of data type combinations before the user writes the user code, reducing the use threshold and further improving the development efficiency.

[0013] In another possible implementation, the complete executable code can be executed in a separate namespace that limits one or more of a type of built-in function, a maximum process time requirement, and a memory usage. In this way, namespace isolation and resource limitation can be implemented when the complete executable code is executed, a more rigorous security boundary is built, dangerous module import and resource abuse are prevented, and system and data security is ensured.

[0014] In a second aspect, the present application provides a visual data processing apparatus, which can include: a data receiving module, which can be used to obtain a data file and store the data file into a temporary directory; a data processing module, which can be used to identify a format of the data file, and invoke a corresponding loading function based on the format of the data file, and the data processing module can also be used to generate a data loading code based on the temporary directory where the data file is stored and the corresponding loading function; a code injection module, which can be used to receive a user code from a client and preposition the data loading code into the user code to generate a complete executable code; and an execution module, which can be used to execute the complete executable code, generate a data visualization result, and send the data visualization result to the client.

[0015] In a possible implementation, the visual data processing apparatus can further include an information extraction module, which can be used to generate a global data object and / or metadata information for the user code based on the data file.

[0016] In a third aspect, the present application provides an electronic device, which includes a processor and a memory, the processor being coupled with the memory; the memory is used to store computer instructions, the computer instructions being loaded and executed by the processor to enable the computer device to implement the method of the first aspect.

[0017] In a fourth aspect, the present application provides a computer readable storage medium, which includes computer execution instructions, when the computer execution instructions run on a computer, enabling the computer to execute the method of the first aspect.

[0018] In a fifth aspect, the present application provides a computer program product, which comprises a computer program, and when the computer program is executed on an electronic device, the electronic device is caused to execute the method of the first aspect.

[0019] The advantages of the second aspect to the fifth aspect are described in the first aspect, and will not be repeated. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 An example application environment of a visual data processing method provided by an embodiment of the present application.

[0021] Figure 2 A flowchart of a visual data processing method provided by an embodiment of the present application.

[0022] Figure 3 A flowchart of information extraction in a visual data processing method provided by an embodiment of the present application.

[0023] Figure 4 An example system architecture provided by an embodiment of the present application.

[0024] Figure 5 A composition diagram of a visual data processing apparatus provided by an embodiment of the present application.

[0025] Figure 6 A composition diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0026] A visual data processing method provided by an embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0027] The term "and / or" in this document merely describes an association relationship of associated objects, and indicates that there can be three relationships, for example, A and / or B can represent three cases of A alone, A and B together, and B alone.

[0028] The terms "first" and "second" and the like in the specification and drawings of the present application are used to distinguish different objects or different processing of the same object, and are not used to describe a specific order of the objects.

[0029] In addition, the terms "include" and "have" and any variations thereof mentioned in the description of the present application are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but can optionally include other steps or units not listed or can optionally include other steps or units inherent to the process, method, product or device.

[0030] It should be noted that in the embodiments of the present application, the words such as "exemplary" or "for example" are used to represent an example, illustration or description. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the words such as "exemplary" or "for example" are intended to present the relevant concept in a specific manner.

[0031] In order to clearly describe the technical solutions of the embodiments of the present application, in the embodiments of the present application, the words "first", "second" and the like are used to distinguish the same or similar items with basically the same function and role, and those skilled in the art can understand that the words "first", "second" and the like are not limited in quantity and execution order.

[0032] In the description of the present application, unless otherwise specified, the meaning of "a plurality of" is two or more.

[0033] At present, data is being generated at an unprecedented speed and scale, and is penetrating into various fields such as scientific research, medical treatment, finance and industry. With the deep integration of data science and visualization technology, data visualization processing has become the core means in the fields of scientific research analysis, business decision, engineering development and the like.

[0034] For data visualization technology, programming visualization tools (such as Jupyter Notebook, Matplotlib) require users to manually write data reading code when implementing data visualization, and users need to handle file paths, format parameters and other details by themselves, which constitutes a significant use barrier for non-technical users. Low-code tools simplify the process through drag-and-drop operations, but completely shield the underlying code and cannot implement complex custom visualization logic.

[0035] To address the aforementioned technical problems, this application provides a data visualization processing method. This method performs data visualization by: acquiring a data file and storing it in a temporary directory; identifying the format of the data file and calling the corresponding loading function based on that format; generating data loading code based on the temporary directory where the data file is stored (e.g., file paths in the temporary directory can be further securely escaped) and the corresponding loading function; receiving user code from the client and pre-loading the data loading code into the user code to generate complete executable code; executing the complete executable code, for example, in a separate namespace to generate a data visualization result and sending the data visualization result to the client. This technical solution solves the problem of cumbersome data loading, allowing users to perform data visualization processing without worrying about the details of data loading, thereby reducing operational costs, improving development efficiency, and achieving a seamless "plug-and-play" experience.

[0036] The embodiments provided in this application will now be described in detail with reference to the accompanying drawings.

[0037] This application provides a method for visualizing data processing, which can be applied to, for example... Figure 1 The example application environment shown. For example... Figure 1 As shown, the application environment includes: a visualization data processing server 101 (hereinafter referred to as server 101), a client device 102 (hereinafter referred to as client 102), and a user 103. Among them, server 101 and client 102 communicate with each other through a network.

[0038] In some embodiments, server 101 may be a server cluster consisting of multiple servers, a single server, a computer, or a processor or processing chip in a server or computer. This application does not limit the specific device form of server 101. Figure 1 The example shown is server 101 as a single server.

[0039] In some embodiments, client 102 can be a device with wireless transceiver capabilities, such as a mobile phone, tablet computer, wearable device, in-vehicle device, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), etc. This application embodiment does not limit the specific device form of client 102. Figure 1The client 102 is exemplified as a notebook computer. The user 103 interacts with the server 101 through the client 102 to realize visual data processing.

[0040] It should be noted that the system architecture described in the embodiments of the present application is for more clearly illustrating the technical solutions of the embodiments of the present application, and does not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, with the evolution of the system architecture, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0041] Figure 2 A flowchart of a visual data processing method provided by the embodiments of the present application is shown in FIG. 2. As shown in FIG. 2, the visual data processing method provided by the present application can be implemented by the server 101, and specifically includes the following steps S201-S206. Figure 2

[0042] S201, the server 101 acquires a data file and stores the data file into a temporary directory.

[0043] When the user wants to perform visual data processing on a certain data file, the user can upload the data file through the client, or the user can instruct the server to acquire the data file from its memory or database.

[0044] For example, in some embodiments, the user 103 can upload the data file through the client 102. In other embodiments, the user 103 can instruct the server 101 to acquire the data file from its memory or other locations (e.g., a database). However, the embodiments of the present application are not limited thereto, and the data file can be acquired from any other location.

[0045] For example, the data file can be a CSV, Excel, JSON, or other format file, but the embodiments of the present application are not limited thereto, and the data file can be any other format file.

[0046] As an example, the user 103 can upload a CSV file through the client 102. For example, the user 103 uploads sales_data.csv, and the content is as follows:

[0047] Then, the server 101 receives the CSV file from the client 102, stores the CSV file into the temporary directory, and generates a unique session ID (e.g., session_123) to associate the operation.

[0048] S202, the server 101 identifies the format of the data file, and calls a corresponding loading function based on the format of the data file. ​

[0049] In some embodiments, the server 101 identifies the format of the data file through a file extension (such as.csv / .xlsx / .json) or file header information, and invokes a corresponding loading function in the Pandas library based on the format of the data file.

[0050] For example, if the format of the data file is CSV, Excel and JSON, the loading function can be read_csv(), read_excel() and read_json() respectively, however, embodiments of the present application are not limited thereto, and the loading function can be any other loading function corresponding to the format of other data files for reading files.

[0051] S203, the server 101 generates a data loading code based on the temporary directory in which the data file is stored and the corresponding loading function.

[0052] It should be noted that in some embodiments, after storing the obtained data file in the temporary directory, the server 101 further obtains the file path of the data file in the temporary directory, and performs security escaping on the file path of the data file in the temporary directory, and generates the data loading code based on the security-escaped file path and the corresponding loading function.

[0053] For example, the obtained file path can be standardized by os.path.abspath() and then escaped by json.dumps(file_path), and only the temporary directory is allowed to be accessed. In other words, a path whitelist is established, and only paths in the path whitelist can be accessed to prevent path injection attacks (such as.. / etc / passwd) and malicious file reading.

[0054] As an example, continuing with the data file sales_data.csv uploaded by user 103 to client 102, after server 101 stores the received data file sales_data.csv in a temporary directory, server 101 can obtain the file path of the data file sales_data.csv in that temporary directory and perform security escaping on the file path (for example, the securely escaped file path is " / tmp / uploads / session_123 / sales_data.csv"). Furthermore, server 101 can automatically identify the format of the temporary file through the file extension (.csv), and based on the format of the temporary file, call the corresponding loading function read_csv() to read the data from the data file in the securely escaped file path (for example, " / tmp / uploads / session_123 / sales_data.csv") to generate data loading code.

[0055] For example, based on the data file sales_data.csv in the example above, the following sample data loading code can be generated: import pandas as pd try: df=pd.read_csv(" / tmp / uploads / session_123 / sales_data.csv")# The file path after safe escaping. In some embodiments, to ensure that the data loading code above can successfully read the data file, the server can simultaneously generate data loading code that "prints a data summary" to help the user 103 debug.

[0056] For example, continuing with the above example, the sample data loading code could also be as follows: import pandas as pd try: df=pd.read_csv(" / tmp / uploads / session_123 / sales_data.csv")# The file path after safe escaping. #Optional: Print data summary to help users debug version 103. print(f"Data loaded successfully, containing {df.shape [1]} columns: {','.join(df.columns)}") except Exception as e: raise ValueError(f"Data loading failed: {str(e)}") S204. Receive user code from the client and load the data into the user code to generate complete executable code.

[0057] In some embodiments, user 103 writes user code on client 102, server 101 receives the user code from client 102 and loads the data code into the user code to generate complete executable code.

[0058] For example, continuing the example above, user 103 directly inputs simplified drawing code on the client side. Example user code written by the user can be shown below: import matplotlib.pyplot as plt plt.bar(df['month'],df['sales amount (ten thousand yuan)']) plt.title('Monthly Sales Comparison') plt.show() Then, user 103 submits drawing code on the client. After clicking submit, server 101 automatically executes subsequent operations. Server 101 receives user code from client 102 and loads data into the user code to generate complete executable code.

[0059] For example, continuing with the above example, the generated complete executable code can be shown below: injected_code=f""" #Automatic data injection loading logic (not visible to user 103) import pandas as pd try: df=pd.read_csv(" / tmp / uploads / session_123 / sales_data.csv")# The file path after safe escaping. #Optional: Print data summary to help users debug version 103. print(f"Data loaded successfully, containing {df.shape [1]} columns: {','.join(df.columns)}") except Exception as e: raise ValueError(f"Data loading failed: {str(e)}") #User code starts {user_code} import matplotlib.pyplot as plt plt.bar(df['month'],df['sales amount (ten thousand yuan)']) plt.title('Monthly Sales Comparison') plt.show() S205. Execute the complete executable code, generate the data visualization result, and send the data visualization result to the client.

[0060] In some embodiments, the complete executable code is executed in a separate namespace that restricts one or more of the types of built-in functions, maximum process time requirements, and memory usage.

[0061] For example, create a separate namespace (such as namespace={"pd":pd,"plt":plt}) to avoid variable pollution and preload necessary libraries.

[0062] In some embodiments, the types of built-in functions in a separate namespace are restricted, such as removing functions like exec / eval / open from the namespace or disabling the execution of functions like exec / eval. In some embodiments, user code access to dangerous modules is restricted, for example, prohibiting user 103 from importing the os / subprocess module.

[0063] In some embodiments, the maximum process time requirement or memory usage for that individual namespace can be limited. For example, code can be executed using subprocess (or Docker containers for a more stringent sandbox), and timeouts can be set via subprocess (e.g., 30 seconds) or CPU / memory quotas can be limited using Docker containers for dynamic resource management to prevent resource exhaustion caused by malicious code.

[0064] However, the embodiments of this application are not limited thereto. For example, in some embodiments, triple protection of path escaping, namespace isolation, and resource restriction can be implemented simultaneously to build a more robust security boundary and effectively intercept gray-box attacks (such as timing analysis and power analysis).

[0065] In some embodiments, server 101 executes the complete executable code, returns the generated image path or binary data to the front-end for rendering, and cleans up temporary files to release resources.

[0066] For example, if the execution of the complete executable code is successful, the image generated by Matplotlib (e.g., session_123.png) is automatically saved to a temporary directory, session_123.png is obtained through file monitoring, and a URL is returned for the client 102 to render; if the execution of the complete executable code fails, error data is returned for the client 102 to display.

[0067] In some embodiments, in order to ensure that the user can use the data object in the code loaded by the server 101 to automatically generate data in subsequent code writing, and to provide the user with initial template code and code prompts, the process of writing code by the user is simplified, the coding difficulty is reduced, and the development efficiency is improved. The above method can further include step S206.

[0068] S206, based on the data file, generating a global data object and / or metadata information for use in the user code.

[0069] As shown in Figure 2 S206 can be performed after S203, however, embodiments of the present application are not limited thereto, in some embodiments, S206 can also be performed after S202.

[0070] Figure 3 A flowchart of information extraction (i.e., generating a global data object and / or metadata information for use in the user code) in a visual data processing method provided by embodiments of the present application. As shown in Figure 3 The information extraction provided by the present application can be implemented by the server 101 described above, and S206 described above can specifically include steps S2061-S2063.

[0071] S2061, reading data in the data file using a corresponding loading function.

[0072] In some embodiments, after the server 101 identifies the format of the data file, the corresponding loading function is called based on the format of the data file to read the data in the data file.

[0073] For example, the loading function can be read_csv(), read_excel(), and read_json(), however, embodiments of the present application are not limited thereto.

[0074] S2062, converting the data in the data file into predetermined format data to generate the global data object.

[0075] In some embodiments, the server 101 can invoke a corresponding loading function (e.g., read_csv()) to convert the data in the data file into predetermined format data to generate the global data object (e.g., df = pd.read_csv(...)).

[0076] Exemplarily, the predetermined format data can be Series or DataFrame. Exemplarily, when the data file contains multiple columns and no column is specified, the loading function returns a DataFrame object by default; when the data file contains only one column, the loading function directly returns a Series object. However, embodiments of the present application are not limited thereto, and the predetermined format data can be any other format data.

[0077] S2063、Optionally, according to the data in the data file, the metadata information is extracted.

[0078] In some embodiments, after the server 101 generates the global data object (e.g., df = pd.read_csv(...)), the metadata information can be extracted according to the data in the data file.

[0079] In some embodiments, the metadata information includes column names, data types, and data shapes. However, embodiments of the present application are not limited thereto, and the metadata information can be any other data information.

[0080] Exemplarily, column names, data types, and data shapes can be extracted through df.columns, df.dtypes, and df.shape. However, embodiments of the present application are not limited thereto, and any other information extraction method can be used to extract the metadata information.

[0081] In some embodiments, the metadata information can be used to provide code prompts to the user to help the user complete the user code when the user edits the user code.

[0082] Exemplarily, when the user 103 edits the code using the code editing module of the client 102, the server 101 generates completion suggestions in real time according to the metadata information (such as column names, data types), for example, when df[' is input, the completion prompt is triggered, and all column name options are automatically popped up to reduce syntax errors.

[0083] Specifically, continuing the above example, the server 101 prompts the column name completion in real time according to the metadata information, and prompts the month / sales (ten thousand yuan) / profit (ten thousand yuan) after the user 103 inputs df['.

[0084] However, embodiments of the present application are not limited thereto. In some embodiments, the global data object and the metadata information can also be used to generate an initial template code of the user code.

[0085] Exemplarily, by analyzing the data type combination through an algorithm, a suitable chart type is recommended, and an initial code template is automatically generated. For example, two numerical columns recommend a scatter plot, and a classification-numerical column recommends a column chart.

[0086] More specifically, the following table shows the correspondence between the column data types (numerical type / category type) of a data set and the suitable chart types automatically recommended:

[0087] Template recommendation algorithm steps: (1) Data type classification Numerical column: int64, float64 type (for mathematical calculation); Category column: object, category type (discrete classification label); Time column: type containing datetime (time series data); (2) Conditional judgment logic According to the combination relationship of data types, the chart type is recommended in priority order: Column chart (classification-numerical comparison) condition: at least 2 numerical columns and 1 category column; Scatter plot (two numerical variable relationship) condition: exactly 2 numerical columns; Pie chart (category proportion) condition: 1 numerical column and at least 1 category column; Line chart (time series trend) condition: at least 1 time column and 1 numerical column. Cross table heat map condition: at least 2 category columns; No recommendation condition: none of the above conditions are met (such as only a single category column or incomplete data types).

[0088] The above describes in detail the visual data processing method provided by the embodiments of the present application. The following introduces the system architecture for executing the above visual data processing method. Figure 4 The system architecture of the embodiments of the present application is shown in the following figure: Figure 4 The system for executing the above visual data processing method can include a server 101 and a client 102. The server 101 includes a data receiving module 401, a data processing module 402, a code injection module 403, an execution module 404, and an information extraction module 405. The client 102 includes a data uploading module 406, a code editing module 407, and a result display module 408.

[0089] The data receiving module 401 is configured to receive the data file uploaded by the user 103 through the data uploading module 406 and store the data file as a temporary file for performing the step S201. Correspondingly, the data uploading module 406 is configured to receive the data file uploaded by the user 103 to the client interface and transmit the data file to the server 101.

[0090] The data processing module 402 is configured to identify the format of the data file, invoke a corresponding loading function based on the format of the data file, and generate a data loading code based on the temporary directory where the data file is stored and the corresponding loading function, for performing the step S202.

[0091] In some embodiments, the data processing module 402 is further configured to securely escape the file path of the data file in the temporary directory, and generate the data loading code based on the securely escaped file path and the corresponding loading function.

[0092] The code injecting module 403 is configured to receive the user code from the client and prepend the data loading code to the user code to generate a complete executable code, for performing the step S203. Correspondingly, the code editing module 407 is configured to support the user 103 to write and / or modify the Python drawing code to form a complete user code and send the complete user code to the server 101.

[0093] The execution module 404 is configured to execute the complete executable code, generate a data visualization result, and send the data visualization result to the client, for performing the step S204. Correspondingly, the result displaying module 408 is configured to display a visualization image based on the data visualization result from the client, such as displaying a generated image URL or binary data.

[0094] The information extracting module 405 is configured to generate a global data object and / or metadata information based on the data file for use by the user code, for performing the step S205.

[0095] In some embodiments, the information extracting module 405 is further configured to read data in the data file using the corresponding loading function, convert the data in the data file into predetermined format data to generate the global data object, and / or extract the metadata information according to the data in the data file.

[0096] In some embodiments, the metadata information includes column names, data types, and data shapes.

[0097] In some embodiments, the metadata information is used to provide code hints to the user to help the user complete the user code when the user edits the user code.

[0098] In some embodiments, the global data object and the metadata information are used to generate the initial template code for the user code.

[0099] In some embodiments, the complete executable code is executed in a separate namespace that restricts one or more of the types of built-in functions, maximum process time requirements, and memory usage.

[0100] In some embodiments, if the execution of the complete executable code is successful, a URL is returned for the front end to display; if the execution of the complete executable code fails, error data is returned to the front end for rendering.

[0101] In some embodiments, this application also provides a visualization data processing apparatus. This visualization data processing apparatus may include one or more functional modules for implementing the visualization data processing method described in the above embodiments.

[0102] For example, Figure 5 This is a schematic diagram illustrating the composition of a visualization data processing device 500 provided in an embodiment of this application. For example... Figure 5 As shown, the visualization data processing device 500 includes: a data receiving module 501, a data processing module 502, a code injection module 503, and an execution module 504. Specifically, the data receiving module 501, the data processing module 502, the code injection module 503, and the execution module 504 are connected to... Figure 4 The data receiving module 401, data processing module 402, code injection module 403, and execution module 404 in the system correspond to each other and are used to implement the same functions as the corresponding functional modules in the system architecture.

[0103] The data receiving module 501 receives a data file from the client 102 and stores it in a temporary directory. The data processing module 502 identifies the format of the data file, calls the corresponding loading function based on the format, and generates data loading code based on the temporary directory where the data file is stored and the corresponding loading function. The code injection module 503 receives user code from the client 102 and pre-inserts the data loading code into the user code to generate complete executable code. The execution module 504 executes the complete executable code, generates a data visualization result, and sends the data visualization result to the client 102.

[0104] In some embodiments, the visualization data processing device 500 further includes: an information extraction module 405, configured to generate global data objects and / or metadata information for use by the user code based on the data file. Figure 5 Not shown in the image.

[0105] In some embodiments, the data processing module 405 is further configured to securely escape the file path of the data file in the temporary directory, and generate the data loading code based on the securely escaped file path and the corresponding loading function.

[0106] In some embodiments, the information extraction module 405 is further configured to read data in the data file using the corresponding loading function, convert the data in the data file into pre-determined format data to generate the global data object, and / or extract the metadata information according to the data in the data file.

[0107] In some embodiments, the metadata information comprises column names, data types, data shapes.

[0108] In some embodiments, the metadata information is used to provide code hints to the user to help the user complete the user code when the user edits the user code.

[0109] In some embodiments, the global data object and the metadata information are used to generate an initial template code of the user code.

[0110] In some embodiments, the complete executable code is executed in a separate namespace that limits one or more of a type of built-in functions, a maximum process time requirement, and a memory usage.

[0111] In the case where the functions of the above-mentioned integrated modules are implemented in the form of hardware, the embodiments of the present application provide a possible structural schematic diagram of the electronic device involved in the above-mentioned embodiments. As shown in the figure, the electronic device 600 comprises a processor 602, a communication interface 603, and a bus 604. Optionally, the electronic device 600 can further comprise a memory 601. Figure 6

[0112] The processor 602 can be various exemplary logical blocks, modules and circuits described in conjunction with the disclosure of the present application. The processor 602 can be a central processing unit, a general purpose processor, a digital signal processor, an application specific integrated circuit, a field programmable gate array or other programmable logic device, transistor logic device, hardware component or any combination thereof. It can implement or execute various exemplary logical blocks, modules and circuits described in conjunction with the disclosure of the present application. The processor 602 can also be a combination of computing functions, such as a combination of one or more microprocessors, a combination of DSP and microprocessor, etc.

[0113] ​The communication interface 603 is configured to connect with other devices through a communication network. The communication network can be an Ethernet, a wireless access network, a wireless local area network (WLAN), etc.

[0114] The memory 601 can be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, an electrically erasable programmable read-only memory (EEPROM), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to this.

[0115] As a possible implementation, the memory 601 can exist independently of the processor 602, and the memory 601 can be connected with the processor 602 through the bus 604, for storing instructions or program codes. When the processor 602 invokes and executes the instructions or program codes stored in the memory 601, the method for processing visual data provided by the embodiments of the present application can be implemented.

[0116] In another possible implementation, the memory 601 can also be integrated with the processor 602.

[0117] The bus 604 can be an extended industry standard architecture (EISA) bus, etc. The bus 604 can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, Figure 6 Only one thick line is used in the figure to represent the bus, but it does not mean that there is only one bus or only one type of bus.

[0118] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above division of functional modules is taken as an example for illustration, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the service calling device is divided into different functional modules to complete all or part of the functions described above.

[0119] The embodiment of the present application further provides a computer readable storage medium. All or part of the processes in the method embodiments can be instructed by computer instructions to complete by related hardware, and the program can be stored in the computer readable storage medium. When the program is executed, the program can include the processes of the method embodiments. The computer readable storage medium can be the memory of any of the foregoing embodiments. The computer readable storage medium can also be an external storage device of the service calling device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the computer readable storage medium can include both the internal storage unit of the service calling device and the external storage device. The computer readable storage medium is used to store the computer program and other programs and data required by the service calling device. The computer readable storage medium can also be used to temporarily store data that has been output or will be output.

[0120] The embodiment of the present application further provides a computer program product, which comprises a computer program, and when the computer program product runs on a computer, the computer program product makes the computer execute any one of the visual data processing methods provided in the embodiments.

[0121] It should be noted that the visual data processing process described in the embodiments of the present application is for more clearly illustrating the technical solutions of the embodiments of the present application, and does not constitute a limitation on the technical solutions provided by the embodiments of the present application.

[0122] It can be seen that the above mainly introduces the solutions provided by the embodiments of the present application from the perspective of the method. In order to realize the above functions, the embodiments of the present application provide corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should easily realize that the modules and algorithm steps of the examples described in combination with the embodiments disclosed in the present application can be realized in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the form of hardware or computer software driven hardware depends on the specific application and design constraints of the technical solutions. Professional technicians can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0123] The embodiments of the present application can divide the functional modules of the visual data processing server according to the above method examples. For example, each functional module can be divided according to each function, or two or more functions can be integrated in one processing module. The above integrated module can be realized in the form of hardware or in the form of a software functional module. Optionally, the division of the modules in the embodiments of the present application is illustrative, and is only a logical functional division. In actual implementation, another division mode can be used.

[0124] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any change or replacement within the technical scope disclosed in the present application should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for visualizing data processing, characterized in that, The method includes: Obtain the data file and store it in a temporary directory; Identify the format of the data file and call the corresponding loading function based on the format of the data file; Based on the temporary directory where the data file is stored and the corresponding loading function, generate data loading code; Receive user code from the client and preload the data loading code into the user code to generate complete executable code; The complete executable code is executed to generate data visualization results, which are then sent to the client.

2. The method according to claim 1, characterized in that, The step of generating data loading code based on the temporary directory where the data file is stored includes: The file path of the data file in the temporary directory is securely escaped; The data loading code is generated based on the file path that has been safely escaped and the corresponding loading function.

3. The method according to claim 1, characterized in that, The method further includes: Based on the data file, generate global data objects and / or metadata information for use by the user code.

4. The method according to claim 3, characterized in that, The step of generating global data objects and / or metadata information for use by the user code based on the data file includes: Use the corresponding loading function to read the data from the data file; Convert the data in the data file into a predetermined format to generate the global data object; and / or Extract the metadata information based on the data in the data file.

5. The method according to claim 3 or 4, characterized in that, The metadata information includes column name, data type, and data shape.

6. The method according to claim 3 or 4, characterized in that, The metadata information is used to provide code hints to the user when the user edits the user code, to help the user complete the user code.

7. The method according to claim 3 or 4, characterized in that, The global data object and the metadata information are used to generate the initial template code for the user code.

8. The method according to any one of claims 1 to 4, characterized in that, The complete executable code is executed in a separate namespace, which restricts one or more of the types of built-in functions, maximum process time requirements, and memory usage.

9. A visualization data processing device, characterized in that, The device includes: A data receiving module, which is used to acquire a data file and store the data file in a temporary directory; The data processing module is used to identify the format of the data file, call the corresponding loading function based on the format of the data file, and generate data loading code based on the temporary directory where the data file is stored and the corresponding loading function. A code injection module is used to receive user code from the client and preload the data loading code into the user code to generate complete executable code; An execution module is used to execute the complete executable code, generate data visualization results, and send the data visualization results to the client.

10. An electronic device, characterized in that, The device includes a processor and a memory, the processor being coupled to the memory; the memory is used to store computer instructions, which are loaded and executed by the processor to enable the computer device to implement the visualization data processing method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes computer-executable instructions that, when executed on a computer, cause the computer to perform the visualization data processing method according to any one of claims 1 to 8.

12. A computer program product, characterized in that, The computer program product includes a computer program that, when run on an electronic device, causes the electronic device to perform the visualization data processing method as described in any one of claims 1 to 8.

Images