Data analysis method and system, electronic equipment, storage medium and program product

Abstract

The invention discloses a data analysis method and system, electronic equipment, a storage medium and a program product, and relates to the field of artificial intelligence technology and data processing. The method is applied to a client side and comprises the steps that a task description text is obtained, and the task description text is used for reflecting data analysis requirements of to-be-analyzed data; the task description text is analyzed through a data analysis system, a data analysis task corresponding to the task description text is constructed, the data analysis task comprises a plurality of task steps, and the task steps are used for conducting data analysis on the to-be-analyzed data according to data analysis requirements; and executing the plurality of task steps by using the data analysis system to obtain a data analysis result of the to-be-analyzed data. According to the method and the device, the technical problem of relatively poor convenience of data analysis of specified data by a user according to own requirements in related technologies is solved.

Description

Technical Field

[0001] The present application relates to the fields of artificial intelligence technology and data processing, and specifically to a data analysis method, system, electronic device, storage medium and program product. Background Art

[0002] In today's digital age, data analysis has become an important basis for decision-making for businesses and individuals. Traditional data analysis tools, such as business intelligence software or data science platforms, although powerful, often have certain usage barriers. They usually require users to have a certain level of data knowledge and programming skills, such as familiarity with SQL (Structured Query Language), understanding of data structures and statistical principles, etc. Sometimes, it is even necessary to master programming languages ​​such as Python to effectively screen, process and analyze data. As a result, these tools are too difficult to use, affecting the convenience of users using these tools for data analysis.

[0003] To address the above-mentioned problems, no effective solutions have been proposed so far. Summary of the Invention

[0004] The embodiments of the present application provide a data analysis method, system, electronic device, storage medium and program product to at least solve the technical problem in the related art that it is not convenient for users to perform data analysis on specified data according to their own needs.

[0005] According to one aspect of an embodiment of the present application, a data analysis method is provided, including: obtaining a task description text, wherein the task description text is used to reflect the data analysis requirements of the data to be analyzed; using a data analysis system to analyze the task description text, and constructing a data analysis task corresponding to the task description text, wherein the data analysis task includes multiple task steps, and the multiple task steps are used to perform data analysis on the data to be analyzed according to the data analysis requirements; using the data analysis system to execute the multiple task steps to obtain data analysis results of the data to be analyzed.

[0006] According to another aspect of an embodiment of the present application, a data analysis method is also provided, including: responding to an input instruction on an operation interface, displaying a task description text on the operation interface, wherein the task description text is used to reflect the data analysis requirements of the data to be analyzed; responding to a processing instruction on the operation interface, displaying a data analysis result on the operation interface, wherein the data analysis result is used to characterize the result obtained by executing a data analysis task using a data analysis system, the data analysis task includes multiple task steps, and the multiple task steps are used to perform data analysis on the data to be analyzed according to the data analysis requirements, and the data analysis task is constructed by the data analysis system by analyzing the task description text.

[0007] According to one aspect of an embodiment of the present application, a data analysis device is provided, including: a text acquisition module, used to acquire a task description text, wherein the task description text is used to reflect the data analysis requirements of the data to be analyzed; a task construction module, used to analyze the task description text using a data analysis system, and construct a data analysis task corresponding to the task description text, wherein the data analysis task includes multiple task steps, and the multiple task steps are used to perform data analysis on the data to be analyzed according to the data analysis requirements; a task execution module, used to execute the multiple task steps using the data analysis system to obtain data analysis results of the data to be analyzed.

[0008] According to another aspect of an embodiment of the present application, a data analysis device is also provided, including: a first display module, used to respond to input instructions acting on an operation interface, and display a task description text on the operation interface, wherein the task description text is used to reflect the data analysis requirements of the data to be analyzed; a second display module, used to respond to processing instructions acting on the operation interface, and display data analysis results on the operation interface, wherein the data analysis results are used to characterize the results obtained by using a data analysis system to perform a data analysis task, the data analysis task includes multiple task steps, and the multiple task steps are used to perform data analysis on the data to be analyzed according to the data analysis requirements, and the data analysis task is constructed by the data analysis system analyzing the task description text.

[0009] According to another aspect of an embodiment of the present application, a data analysis system is also provided, including: a first process module, used to execute any of the above methods; a second process module, which is communicated with the first process module through a preset mechanism, and is used to render and output the data analysis results of the data to be analyzed.

[0010] According to another aspect of the embodiments of the present application, an electronic device is provided, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods in the various embodiments of the present application when running.

[0011] According to another aspect of the embodiments of the present application, an electronic device is also provided, including: a memory storing an executable program; a processor connected to the memory via a bus, and configured to run the program, wherein the method of each embodiment of the present application is executed when the program is running.

[0012] According to another aspect of an embodiment of the present application, a computer-readable storage medium is also provided, which includes a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute the methods in various embodiments of the present application.

[0013] According to another aspect of the embodiments of the present application, a computer program product is further provided, including a computer program, which implements the methods in various embodiments of the present application when executed by a processor.

[0014] According to another aspect of an embodiment of the present application, a computer program product is also provided, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method in each embodiment of the present application is implemented.

[0015] According to another aspect of the embodiments of the present application, a computer program is further provided, which implements the methods in various embodiments of the present application when executed by a processor.

[0016] In an embodiment of the present application, a method is adopted in which a task description text is obtained; the task description text is analyzed by a data analysis system to construct a data analysis task corresponding to the task description text; and the data analysis system is used to execute the multiple task steps to obtain data analysis results of the data to be analyzed. By utilizing the data analysis system deployed on the client, data analysis tasks that can meet the data analysis needs of the user are independently formulated and executed based on the task description text. This can reduce the difficulty of the user in performing data analysis on the data to be analyzed while ensuring the confidentiality and security of the data analysis process, improve the convenience of the data analysis process, and at the same time improve the controllability of the constructed data analysis task, ensure the efficiency and accuracy of the data analysis task during execution, and thus solve the technical problem in the related art that it is less convenient for users to perform data analysis on specified data according to their own needs.

[0017] It is easy to notice that the above general description and the following detailed description are merely for the purpose of exemplifying and explaining the present application, and do not constitute a limitation of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0019] Figure 1 is a schematic diagram of a data analysis scenario according to an embodiment of the present application;

[0020] Figure 2 is a flow chart of a data analysis method according to an embodiment of the present application;

[0021] Figure 3 is a schematic diagram illustrating a data analysis process according to an embodiment of the present application;

[0022] Figure 4 is a flow chart of another data analysis method according to an embodiment of the present application;

[0023] Figure 5 is a structural block diagram of a data analysis device according to an embodiment of the present application;

[0024] Figure 6 is a structural block diagram of another data analysis device according to an embodiment of the present application;

[0025] Figure 7 is a structural block diagram of a data analysis system according to an embodiment of the present application;

[0026] Figure 8 is a structural block diagram of a data analysis system according to an embodiment of the present application;

[0027] Figure 9 is a schematic diagram of a chat analysis page according to an embodiment of the present application;

[0028] Figure 10 is a schematic diagram of a tool configuration page according to an embodiment of the present application;

[0029] Figure 11 This is a schematic diagram of a new plug-in area according to an embodiment of the present application;

[0030] Figure 12 is a structural block diagram of a computing device according to an embodiment of the present application;

[0031] Figure 13 This is a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0034] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0035] The technical solution provided in this application is mainly implemented using a deep learning model. The deep learning model can be widely used in natural language processing (NLP), computer vision, speech processing and other fields. Specifically, it can be applied to computer vision tasks such as visual question answering (VQA), image description (IC), image generation, etc. It can also be widely used in natural language processing tasks such as text-based sentiment classification, text summary generation, and machine translation. Therefore, the main application scenarios of this application include but are not limited to digital assistants, intelligent robots, search, online education, office software, e-commerce, intelligent design, etc. In the embodiments of this application, data processing through a data analysis model in a data analysis scenario is used as an example for explanation.

[0036] First, some nouns or terms that appear in the description of the embodiments of the present application are subject to the following interpretations:

[0037] Data analysis: refers to the process of collecting, processing, organizing and interpreting data to extract useful information and support decision making by using statistics, mathematics, computer programming and other techniques.

[0038] Agent: refers to an entity that can perceive and take actions to achieve a certain goal in a certain environment. It can be a software program, a robot, or any system with autonomous decision-making capabilities.

[0039] Client: An application installed on a personal computer or workstation that interacts with the user through a graphical user interface (GUI) and runs on the local device. Clients are typically used to access remote servers or services, processing data and tasks without requiring the user to use a web browser directly.

[0040] MCP protocol: Model Context Protocol, model context communication protocol, used for standardized interaction between different service components.

[0041] Sandbox environment: An isolated execution environment (based on container technology) used to run untrusted code.

[0042] Data source plug-in: An MCP service component that implements access capabilities to specific data sources.

[0043] Visualization plug-in: An MCP service component that converts analysis results into visual charts.

[0044] According to an embodiment of the present application, a data analysis method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0045] The technical solution provided in the embodiment of the present application can adopt a deep learning model with a relatively large model parameter scale, such as a large model containing billions or even more model parameters. Among them, the large model is only an example of a model. The embodiment of the present application does not limit the number of model parameters supported by the deep learning model adopted, with the goal of meeting actual needs. The deep learning model involved in the embodiment of the present application can be a language model (LanguageMode, LM) or a multimodal model (MultimodalModel, MM) based on artificial intelligence.

[0046] Considering the limited computing resources of the mobile terminal, the above method provided in the embodiment of the present application can be applied to Figure 1 The application scenarios shown are not limited to these. Figure 1 is a schematic diagram of a data analysis scenario according to an embodiment of the present application. Figure 1In the illustrated application scenario, the deep learning model can be directly deployed on the client device 10. The client device 10 can interact with the user via a graphical user interface. The user can input task description text into the client device 10 via the graphical user interface. The corresponding client can then locally perform data analysis on the specified data according to the task description text input by the user, thereby avoiding the risk of data leakage generated during the data analysis process. The client device 20 may include, but is not limited to, a smartphone, tablet computer, laptop computer, PDA, personal computer, smart home device, in-vehicle device, etc. Corresponding to the client device 10, at least one server 20 may also be provided, connected to the client device 10 via a local area network, wide area network, the Internet, or other type of data network, to ensure that the client device 10 can fully obtain the data required for analysis during the data analysis process, thereby meeting the user's data analysis needs.

[0047] In an embodiment of the present application, a system consisting of a client device and a server can perform the following steps: the client device executes to obtain a task description text; uses a data analysis system to analyze the task description text and construct a data analysis task corresponding to the task description text; uses the data analysis system to execute multiple task steps to obtain data analysis results for the data to be analyzed. The server executes to receive a data acquisition request sent by the client; and based on the data acquisition request, returns the target data to the client, wherein the target data includes the data to be analyzed.

[0048] It should be noted that, with the rapid development of high-performance computing units, in other application scenarios, the above method provided in the embodiment of the present application can also be applied to the model all-in-one machine. In an optional embodiment, a variety of models are built into the model all-in-one machine, and the user can choose to adjust with a model as needed to obtain the user's own model, so that the high-performance computing unit built into the model all-in-one machine can directly call the adjusted model to execute the above method provided in the embodiment of the present application. In another optional embodiment, a trained model is built into the deep learning model all-in-one machine, so that the high-performance computing unit built into the model all-in-one machine can directly call the model to execute the above method provided in the embodiment of the present application.

[0049] Furthermore, when users need to train their own models, they can upload their own datasets through the client. This dataset is then sent to the server, which then adjusts the pre-trained model using the dataset to obtain the user's own model, which is then deployed in the client's production environment. To facilitate user model adjustment needs, the server provides a complete set of adjustment tools, development frameworks, and processes, supporting multiple adjustment strategies, making the adjusted model better suited to different application fields and highly customized.

[0050] Under the above operating environment, this application provides Figure 2 The data analysis method shown is applied in the client. It should be noted that in order to avoid data leakage and at the same time ensure that the user's needs for analyzing data in different scenarios are met, the above-mentioned data analysis method can be applied only in the client. For example, a local sandbox environment can be deployed in the client, and the corresponding data analysis system can be configured in the environment to use the data analysis system to execute the above-mentioned data analysis method. Correspondingly, the user can freely set plug-ins such as data analysis models and data analysis tools in the data analysis system, so that the data analysis system can flexibly and efficiently implement data analysis operations to meet user needs. In this way, combining artificial intelligence technology with data analysis tools in the data analysis system not only simplifies the data analysis process and improves analysis efficiency, but also ensures the security of data processing and the accuracy of the results, which can provide users with a more personalized and intelligent analysis experience to a large extent.

[0051] To facilitate understanding of the above data analysis method, Figure 2 This is a flow chart of a data analysis method according to an embodiment of the present application. Figure 2 As shown, the method may include the following steps:

[0052] Step S202: Obtain task description text.

[0053] The task description text is used to reflect the data analysis requirements for the data to be analyzed.

[0054] The task description text may refer to text input by a user that can reflect the user's data analysis requirements for the data to be analyzed.

[0055] In an optional solution of this embodiment, in order to accurately understand the user's data analysis needs, such as analyzing the development trend of data, comparing the differences between data, and judging the correlation between data, the data analysis system can first obtain a task description text that can reflect the user's analysis needs for the data to be analyzed, and then analyze the data to be analyzed based on the task description text, thereby improving the accuracy of subsequent analysis of the data to be analyzed. The data analysis system can be an intelligent agent that can be used to process the received task description text. In order to facilitate users to express their own data analysis needs, the above-mentioned task description text can be presented in the form of natural language, that is, users can directly enter their own data analysis needs in the graphical user interface provided by the client according to daily language expression to form the above-mentioned task description text. For example, if the user wants to know the visit trend of website A in the past month, the user can enter "Analyze the visit trend of website A in the past month" in the graphical user interface. The corresponding data analysis system can use this description text to clarify the user's needs for the data range, time window, indicator type, data trend, etc. of the analyzed data. For ease of understanding, the data range here can be "limited to the data of website A", the time window can be "the past month", and the indicator type can be "website visits". At the same time, the data trend can reflect that the user not only wants to know the specific number of visits, but also needs to know how the visits changed during this period, such as whether there is an upward, downward or fluctuating trend.

[0056] Step S204: Analyze the task description text using the data analysis system to construct a data analysis task corresponding to the task description text.

[0057] The data analysis task includes multiple task steps, and the multiple task steps are used to perform data analysis on the data to be analyzed according to data analysis requirements.

[0058] In an optional solution of this embodiment, considering that only understanding the user's data analysis needs may still result in a rough data analysis process, such as not cleaning the data to be analyzed, not using appropriate data analysis tools, not evaluating and adjusting the analysis results, etc., which may lead to the inability to obtain satisfactory data analysis results for the user. Therefore, after obtaining the above-mentioned task description text, the data analysis system can further analyze the task description text and construct a corresponding data analysis task to formulate a detailed and reasonable data analysis strategy, thereby ensuring that the results obtained from the subsequent analysis of the data to be analyzed can largely meet the user's data analysis needs. In order to be able to well perform the various details that may be involved in the data analysis process, such as removing invalid data and selecting appropriate data analysis tools, the constructed data analysis task can include multiple task steps. The corresponding data analysis system can perform data analysis on the data to be analyzed based on the user's data analysis needs according to these task steps, thereby ensuring that the obtained data analysis results can meet the user's needs. Among them, the multiple task steps in the constructed data analysis task can include but are not limited to: obtaining and integrating the data to be analyzed, cleaning and converting the data to be analyzed, selecting analysis tools, evaluating analysis results, integrating analysis results, etc.

[0059] Step S206: Utilize the data analysis system to execute multiple task steps to obtain data analysis results of the data to be analyzed.

[0060] In one optional solution of this embodiment, after developing a corresponding data analysis task based on the task description text entered by the user, the data analysis system can then execute the multiple task steps included in the data analysis task to analyze the data to be analyzed according to these multiple task steps, thereby obtaining data analysis results that meet the user's needs. To facilitate user viewing and understanding, the data analysis system can also generate corresponding visualization charts, interactive reports, and other information based on the obtained data analysis results, and present them to the user through a graphical user interface.

[0061] In an embodiment of the present application, a method is adopted in which a task description text is obtained; the task description text is analyzed by a data analysis system to construct a data analysis task corresponding to the task description text; and the data analysis system is used to execute the multiple task steps to obtain data analysis results of the data to be analyzed. By utilizing the data analysis system deployed on the client, data analysis tasks that can meet the data analysis needs of the user are independently formulated and executed based on the task description text. This can reduce the difficulty of the user in performing data analysis on the data to be analyzed while ensuring the confidentiality and security of the data analysis process, improve the convenience of the data analysis process, and at the same time improve the controllability of the constructed data analysis task, ensure the efficiency and accuracy of the data analysis task during execution, and thus solve the technical problem in the related art that it is less convenient for users to perform data analysis on specified data according to their own needs.

[0062] In an embodiment of the present application, a data analysis system is configured with a data analysis model; the data analysis system is used to analyze a task description text and construct a data analysis task corresponding to the task description text, including: performing semantic extraction on the task description text to obtain text semantic features of the task description text, wherein the text semantic features are used to reflect data analysis requirements; based on the text semantic features, determining the data source information and application scenarios corresponding to the data to be analyzed, wherein the data source information is used to reflect the validity of at least one data source related to the task description text; inputting the text semantic features, data source information and application scenarios into the data analysis model, and generating a data analysis task using the data analysis model.

[0063] The aforementioned data source information may refer to information about the data source related to the task description text, including but not limited to: the data source structure, field meanings, physical location of the data, and data query permissions. The data source may store data to be analyzed for data analysis. The aforementioned application scenarios may refer to scenarios related to the task description text, including but not limited to e-commerce, finance, and logistics. The meaning and usage rules of the data may vary in different application scenarios.

[0064] In an optional solution of this embodiment, in order to accurately understand the user's data analysis needs through the task description text and formulate a reasonable data analysis task, the above-mentioned data analysis model can be configured in the data analysis system. When constructing the data analysis task, the data analysis system can first perform semantic extraction on the acquired task description text to obtain text semantic features that can reflect the above-mentioned data analysis needs, and then determine the data source information and application scenarios related to the user's data analysis needs based on the text semantic features. Through the data source information, the data analysis system can accurately determine the validity of different data sources, the access method of the valid data sources, and other information, so that the data analysis system can accurately obtain the data to be analyzed related to the user's data analysis needs. Through the application scenario, the data analysis system can accurately determine the data meaning and usage rules of different data that may need to be used, thereby avoiding the situation where the analysis data is processed according to the wrong data meaning and usage rules, resulting in errors in the obtained data analysis results, thereby ensuring the accuracy of the obtained data analysis results. Based on this, after determining the text semantic features of the task description text, as well as the data source information and application scenarios corresponding to the data to be analyzed, the data analysis system can input the text semantic features, data source information and application scenarios into the data analysis model to formulate reasonable data analysis tasks.

[0065] In an embodiment of the present application, a data analysis model includes: a data source determination module, a knowledge base determination module, an external tool determination module and an analysis task generation module; text semantic features, data source information and application scenarios are input into the data analysis model, and a data analysis task is generated using the data analysis model, including: based on the data source information, using the data source determination module to determine a target data source from at least one data source, wherein the target data source includes the data to be analyzed; based on the application scenario, using the knowledge base determination module to determine a target knowledge base from at least one knowledge base, wherein the target knowledge base includes the data meaning and data usage rules corresponding to the data to be analyzed in the application scenario; based on the text semantic features and the data to be analyzed, using the external tool determination module to select at least one second external tool from multiple first external tools, and construct an external tool library based on the at least one second external tool, wherein the at least one second external tool is used to represent an external tool that can perform data processing on the data to be analyzed according to data analysis requirements; inputting the data source information, target knowledge base and external tool library corresponding to the target data source into the analysis task generation module, and using the analysis task generation module to generate a data analysis task.

[0066] In an optional solution of this embodiment, in order to ensure the rationality of the formulated data analysis task, the above-mentioned data source determination module, knowledge base determination module, external tool selection module and analysis task generation module can be set in the data analysis model, wherein, through the data source determination module, the data analysis system can determine the target data source that can meet the user's data analysis needs, through the knowledge base determination module, the data analysis system can determine the meaning and usage rules of different data under the corresponding target data source, through the external tool selection module, the data analysis system can select a suitable external tool for analyzing the data under the target data source, and through the analysis task construction module, the data analysis system can formulate a reasonable data analysis task to ensure that the obtained data analysis results can meet the user's data analysis needs. Based on this, when constructing a data analysis task, the data analysis system can first analyze at least one data source using the data source determination module based on the acquired data source information to determine the validity of different data sources and the degree of relevance of different data sources to the user's data analysis needs. Based on the analyzed degree of relevance and validity, the data analysis system can determine a suitable target data source from at least one data source to obtain accurate data to be analyzed from the target data source. While determining the target data source, the data analysis system can also use the knowledge base determination module to determine the target knowledge base corresponding to the task description text from at least one knowledge base based on the determined application scenario. Through the target knowledge base, the data analysis system can determine the data meaning and data usage rules of the data to be analyzed in the corresponding application scenario, thereby ensuring the accuracy of subsequent data analysis of the data to be analyzed. After determining the corresponding target data source and target knowledge base, the data analysis system can also use the external tool determination module to select at least one second external tool from multiple first external tools to construct a corresponding external tool library based on the text semantic features of the task description text and the data to be analyzed obtained from the target data source, so that the data analysis system can select a suitable second external tool from the external tool library during the data analysis of the data to be analyzed, and perform data processing on the data to be analyzed according to the user's data analysis requirements, thereby ensuring the accuracy of the data analysis results obtained by analyzing the data to be analyzed. Finally, after determining the target data source, application scenario and at least one second external tool, the data analysis system can input the data source information corresponding to the target data source, the target knowledge base and the external tool library into the analysis task generation module, so as to use the analysis task generation module to generate a reasonable data analysis task, thereby ensuring the accuracy of the analysis of the analysis data according to the data analysis task.

[0067] In an embodiment of the present application, a data analysis system is used to execute multiple task steps to obtain data analysis results of the data to be analyzed, including: based on the task steps, selecting the target external tool corresponding to the task steps from the external tool library, wherein different task steps correspond to different target external tools, and the target external tool is used to perform data processing on the data to be analyzed in the corresponding task steps; executing the task steps based on the target external tool to obtain the step execution results corresponding to the task steps; in response to the completion of the execution of multiple task steps, summarizing the step execution results corresponding to the multiple task steps to obtain the data analysis results of the data to be analyzed.

[0068] In an optional solution of this embodiment, when executing multiple task steps, the data analysis system can first select the target external tools required for executing different task steps from the aforementioned external tool library. Using the target external tools, the data analysis system can perform data processing on the data to be analyzed in the corresponding task steps to meet the user's data analysis needs. After selecting the target external tools, the data analysis system can execute the corresponding task steps according to the selected target external tools to obtain the step execution results corresponding to the task steps. Finally, after all task steps are executed, the data analysis system can summarize the step execution results corresponding to the multiple task steps to obtain the data analysis results of the data to be analyzed.

[0069] In an optional solution of this embodiment, considering that when the data analysis task is relatively complex, the existing external tools in the external tool library may not meet the execution standards of the corresponding task steps, resulting in the corresponding task steps being unable to be executed, affecting the final data analysis results. At this time, in order to be able to smoothly execute the corresponding task steps, the data analysis system can generate a corresponding code that can execute the task steps for the task steps, and use the generated code to build the above-mentioned target external tool.

[0070] In an embodiment of the present application, based on the task step, a target external tool corresponding to the task step is selected from an external tool library, including: obtaining the step identifier of the task step and the data characteristics of the data to be analyzed; based on the step identifier, selecting at least one third external tool from the external tool library, wherein the at least one third external tool is used to characterize the external tool that can execute the task step; evaluating the at least one third external tool based on the data characteristics to obtain a tool evaluation result, wherein the tool evaluation result is used to reflect the correlation between the at least one third external tool and the data to be analyzed; based on the tool evaluation result, selecting the target external tool corresponding to the task step from the at least one third external tool.

[0071] The aforementioned step identifiers can be used to reflect the types of external tools that can be used in the corresponding task steps. Tool types may include, but are not limited to, data cleaning tools, data query tools, data source plug-ins, and visualization plug-ins. The aforementioned data characteristics may include, but are not limited to, data type, data volume, data distribution, and structure. Based on these data characteristics, the data analysis system can select appropriate external tools for processing the data to be analyzed.

[0072] In an optional solution of this embodiment, when selecting a target external tool, the data analysis system can first determine the step identifier of the task step and the data characteristics of the data to be analyzed, and then, based on the step identifier, first select at least one third external tool from the external tool library that can execute the task step, and then use the data characteristics of the data to be analyzed to evaluate the selected at least one third external tool to determine the correlation between different third external tools and the data to be analyzed, thereby obtaining the above-mentioned tool evaluation result, wherein the correlation between different third external tools can be determined from multiple aspects such as the processing accuracy and processing efficiency of the third external tool, and the specific details are not limited. After obtaining the tool evaluation results corresponding to different third external tools, the data analysis system can further select a suitable target external tool from at least one third external tool based on the tool evaluation results to ensure the subsequent execution capability of the target external intermediate execution corresponding to the task step.

[0073] In an embodiment of the present application, a task step is executed based on a target external tool to obtain a step execution result corresponding to the task step, including: obtaining a structured query statement corresponding to the target external tool; detecting the query operation that can be executed by the structured query statement to obtain a statement detection result, wherein the statement detection result is used to characterize whether the query operation is safe; in response to the statement detection result characterizing that the query operation is safe, the task step is executed based on the target external tool to obtain a step execution result corresponding to the task step.

[0074] In an embodiment of the present application, the above method also includes: in response to the statement detection result indicating that the query operation is unsafe, based on the step identifier of the task step and the data characteristics of the data to be analyzed, re-selecting a new external tool corresponding to the task step from the external tool library, and executing the task step based on the new external tool if the re-selected new external tool is safe, to obtain the step execution result corresponding to the task step.

[0075] In an optional solution of this embodiment, taking into account that some external tools may have security risks, for example, for data query tools such as SLS (a log service) and ODPS (a data processing service), there may be a risk of modifying or deleting data when applied. Therefore, when using the target external tool to perform the corresponding task steps, the data analysis system can first detect the target external tool currently in use to avoid data modification or deletion due to security risks in the target external tool. Taking into account that in the process of data query and analysis, the structured query statement corresponding to the data query tool, such as the SQL query statement, usually only performs SELECT operations, and does not perform other operations that may modify or delete data, such as UPDATE operations, DELETE operations, etc. Otherwise, it can be considered that the structured query statement corresponding to the data query tool may be abnormal, and the corresponding data query tool may also have security risks. Based on this, when executing the task steps, the data analysis system can first obtain the structured query statement corresponding to the target external tool, such as an SQL query statement, and analyze the obtained structured query statement to determine the query operations that the structured query statement can currently implement, and then detect the analyzed query operations to obtain the corresponding statement detection results to determine whether the analyzed query operations are safe. For example, if the analyzed query operation only includes the SELECT operation, then the query operation can be considered safe, and the corresponding structured query statement used to implement the query operation is also safe, and then the target external tool currently in use can be considered safe. At this time, the data analysis system can use the target external tool to execute the corresponding task steps to obtain the step execution results corresponding to the task steps; if the analyzed query operation not only includes the SELECT operation, but also includes operations such as UPDATE and DELETE that may pose security risks, then the query operation can be considered unsafe, and the corresponding structured query statement used to implement the query operation is also unsafe. At this time, the data analysis system can, based on the step identifier of the task step and the data characteristics of the data to be analyzed, re-select the new external tool corresponding to the task step from the external tool library according to the above process, and use the new external tool to execute the corresponding task step if the re-selected new external tool is safe, thereby obtaining the corresponding step execution results.

[0076] In an optional solution of this embodiment, considering that it may be necessary to execute the structured query statement when detecting the query operations that can be implemented by the structured query statement, and in order to avoid the damage to the data to be analyzed due to the security risks of the structured query statement, when executing the structured query statement, the data analysis system can also use CTE (Common Table Expression) technology to construct a temporary result data set, and then use the temporary result data set to determine the query operations of the structured query statement, thereby ensuring the data security of the data to be analyzed and avoiding data modification or deletion.

[0077] In an embodiment of the present application, a task step is executed based on a target external tool to obtain a step execution result corresponding to the task step, including: detecting the code used by the target external tool to determine the code type of the code; in response to the code type being a preset code type, sending the target external tool to a preset execution environment, and in the preset execution environment, executing the task step based on the target external tool to obtain a step execution result corresponding to the task step.

[0078] In an optional scheme of this embodiment, considering that some codes need to access the network, disk file system or other sensitive system resources, and such codes are likely to have the risk of information leakage, therefore, when actually executing the task steps, the data analysis system can also detect the code used by the target external tool to determine the code type of the code. Correspondingly, if the code type is a preset code type, that is, there may be a risk of information leakage when executing the task steps using the target external tool, then the data analysis system can send the target external tool to a preset execution environment, such as a sandbox environment such as a virtual machine or container, so as to execute the task steps through the target external tool in the preset execution environment, thereby obtaining the step execution results while ensuring the information security of the step execution process.

[0079] In an embodiment of the present application, after executing the task step based on the target external tool and obtaining the step execution result corresponding to the task step, the above method includes: converting the format of the step execution result corresponding to the task step to obtain a conversion execution result; inputting the task description text and the conversion execution result into the data analysis model, and using the data analysis model to evaluate the step execution result to obtain a step evaluation result, wherein the step evaluation result is used to determine whether the step execution result corresponding to the task step can meet the data analysis requirements; in response to the step evaluation result being that the step execution result can meet the data analysis requirements, determining that the task step execution is completed.

[0080] In an embodiment of the present application, the above method also includes: in response to the step evaluation result being that the step execution result corresponding to the task step cannot meet the data analysis requirements, based on the step identifier of the task step and the data characteristics of the data to be analyzed, re-selecting a new external tool corresponding to the task step from the external tool library, and using the re-selected new external tool to execute the task step to obtain a new step execution result corresponding to the task step, until the new external tool corresponding to the task step can meet the data analysis requirements.

[0081] In an optional solution of this embodiment, considering that even if the selected target external tool has a high correlation with the task step, it may still be possible that the task execution result obtained by using the target external tool does not meet the user's data analysis needs. Therefore, after using the target external tool to execute the corresponding task step and obtain the step execution result, the data analysis system can further evaluate the obtained step execution result to determine whether the step execution result can meet the user's data analysis needs. Based on this, the data analysis system can first convert the format of the step execution result corresponding to the task step to obtain a converted execution result that can be quickly evaluated, and then input the task description text and the step execution result into the data analysis model to use the data analysis model to evaluate the step execution result and determine whether the step execution result can meet the user's data analysis needs, thereby obtaining the above-mentioned step evaluation result. Correspondingly, if the step evaluation result shows that the step execution result can meet the user's data analysis needs, the data analysis system can determine that the corresponding task step has been completed; if the step evaluation result shows that the step execution result does not meet the user's data analysis needs, the data analysis system can re-select the new external tool corresponding to the task step from the external tool library based on the step identifier of the task step and the data characteristics of the data to be analyzed, and use the re-selected new external tool to execute the corresponding task step according to the above process to obtain the new step execution result corresponding to the task step, until the selected new external tool can meet the user's data analysis needs.

[0082] To facilitate understanding of the above data analysis process, Figure 3 is a schematic diagram of a data analysis process according to an embodiment of the present application, such as Figure 3As shown, when performing data analysis, the user can first input the corresponding task description text in the form of natural language through the graphical user interface provided by the client, and then the data analysis system deployed in the client can construct the corresponding data analysis task based on the received task description text. The data analysis task can include multiple task steps. Correspondingly, when executing the data analysis task, the data analysis system can first select appropriate external tools for different task steps, and then use the external tools to execute the corresponding task steps to obtain the corresponding step execution results. After obtaining the step execution results, the data analysis system can also evaluate the step execution results to determine whether the results can meet the user's data analysis needs. If they do, then after the execution of multiple task steps is completed, the data analysis system can summarize the step execution results of multiple task steps to obtain the corresponding data analysis results; if not, the data analysis system can reselect new external tools until the newly obtained step execution results can meet the user's data analysis needs.

[0083] It should be noted that for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0084] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0085] According to an embodiment of the present application, another data analysis method is provided. Figure 4 is a flow chart of another data analysis method according to an embodiment of the present application, such as Figure 4 As shown, the method includes the following steps:

[0086] Step S402 : In response to an input instruction on the operation interface, a task description text is displayed on the operation interface.

[0087] The task description text is used to reflect the data analysis requirements for the data to be analyzed.

[0088] Step S404 , responding to the processing instruction on the operation interface, and displaying the data analysis result on the operation interface.

[0089] Among them, the data analysis results are used to characterize the results obtained by using the data analysis system to perform data analysis tasks. The data analysis tasks include multiple task steps, and the multiple task steps are used to perform data analysis on the analysis data according to the data analysis requirements. The data analysis tasks are constructed by the data analysis system by analyzing the task description text.

[0090] In an optional solution of this embodiment, during the data analysis process, if an input instruction acting on the operation interface is detected, the data analysis system can first display the task description text entered by the user on the operation interface to determine the user's data analysis requirements for the data to be analyzed. After detecting the processing instruction acting on the operation interface, the data analysis system can analyze the task description text to construct a corresponding data analysis task. The data analysis task can include multiple task steps. The data to be analyzed is then analyzed according to these multiple task steps to obtain corresponding step execution results. Finally, these multiple step execution results are summarized to obtain the data analysis results corresponding to the data to be analyzed. The data analysis system can finally output the data analysis results on the operation interface for user viewing.

[0091] According to an embodiment of the present application, a data analysis device for implementing the above data analysis method is also provided. Figure 5 is a structural block diagram of a data analysis device according to an embodiment of the present application. Figure 5 As shown, the device includes: a text acquisition module 502, a task construction module 504 and a task execution module 506.

[0092] Among them, the text acquisition module 502 is used to obtain the task description text, wherein the task description text is used to reflect the data analysis requirements of the data to be analyzed; the task construction module 504 is used to use the data analysis system to analyze the task description text and construct the data analysis task corresponding to the task description text, wherein the data analysis task includes multiple task steps, and the multiple task steps are used to perform data analysis on the data to be analyzed according to the data analysis requirements; the task execution module 506 is used to use the data analysis system to execute multiple task steps to obtain the data analysis results of the data to be analyzed.

[0093] In an embodiment of the present application, the data analysis system is configured with a data analysis model; the task construction module 504 includes: a semantic extraction unit, which is used to perform semantic extraction on the task description text to obtain the text semantic features of the task description text, wherein the text semantic features are used to reflect the data analysis requirements; an information determination unit, which is used to determine the data source information and application scenarios corresponding to the data to be analyzed based on the text semantic features, wherein the data source information is used to reflect the validity of at least one data source related to the task description text; a task construction unit, which is used to input the text semantic features, data source information and application scenarios into the data analysis model, and use the data analysis model to generate a data analysis task.

[0094] In an embodiment of the present application, the data analysis model includes: a data source determination module, a knowledge base determination module, an external tool determination module and an analysis task generation module; the task construction unit is also used to: based on the data source information, use the data source determination module to determine the target data source from at least one data source, wherein the target data source includes the data to be analyzed; based on the application scenario, use the knowledge base determination module to determine the target knowledge base from at least one knowledge base, wherein the target knowledge base includes the data meaning and data usage rules corresponding to the data to be analyzed in the application scenario; based on the text semantic features and the data to be analyzed, use the external tool determination module to select at least one second external tool from multiple first external tools, and construct an external tool library based on the at least one second external tool, wherein the at least one second external tool is used to represent an external tool that can perform data processing on the data to be analyzed according to data analysis requirements; input the data source information, target knowledge base and external tool library corresponding to the target data source into the analysis task generation module, and use the analysis task generation module to generate a data analysis task.

[0095] In an embodiment of the present application, the task execution module 506 includes: a tool selection unit, which is used to select the target external tool corresponding to the task step from the external tool library based on the task step, wherein different task steps correspond to different target external tools, and the target external tool is used to perform data processing on the data to be analyzed in the corresponding task step; a step execution unit, which is used to execute the task step based on the target external tool and obtain the step execution result corresponding to the task step; and a result summary unit, which is used to summarize the step execution results corresponding to the multiple task steps in response to the completion of the execution of multiple task steps, and obtain the data analysis result of the data to be analyzed.

[0096] In an embodiment of the present application, the tool selection unit is also used to: obtain the step identifier of the task step and the data characteristics of the data to be analyzed; based on the step identifier, select at least one third external tool from the external tool library, wherein the at least one third external tool is used to characterize the external tool that can execute the task step; evaluate the at least one third external tool based on the data characteristics to obtain a tool evaluation result, wherein the tool evaluation result is used to reflect the correlation between the at least one third external tool and the data to be analyzed; based on the tool evaluation result, select the target external tool corresponding to the task step from the at least one third external tool.

[0097] In an embodiment of the present application, the step execution unit is also used to: obtain the tool type of the target external tool; in response to the tool type being a preset tool type, verify the target external tool to obtain a tool verification result, wherein the tool verification result is used to characterize whether the target external tool is safe; in response to the tool verification result being that the target external tool is safe, execute the task step based on the target external tool to obtain a step execution result corresponding to the task step.

[0098] In an embodiment of the present application, the above-mentioned device also includes: a re-evaluation module, which is used to, in response to the tool verification result that the target external tool is unsafe, re-select a new external tool corresponding to the task step from the external tool library based on the step identifier of the task step and the data characteristics of the data to be analyzed, and if the re-selected new external tool is safe, execute the task step based on the new external tool to obtain the step execution result corresponding to the task step.

[0099] In an embodiment of the present application, the step execution unit is also used to: detect the code used by the target external tool and determine the code type of the code; in response to the code type being a preset code type, send the target external tool to a preset execution environment, and in the preset execution environment, execute the task steps based on the target external tool to obtain the step execution results corresponding to the task steps.

[0100] In an embodiment of the present application, the above-mentioned device also includes: a result conversion module, which is used to convert the format of the step execution result corresponding to the task step to obtain a conversion execution result; a result evaluation module, which is used to input the task description text and the conversion execution result into the data analysis model, and use the data analysis model to evaluate the step execution result to obtain a step evaluation result, wherein the step evaluation result is used to determine whether the step execution result corresponding to the task step can meet the data analysis requirements; a step completion module, which is used to determine that the task step execution is completed in response to the step evaluation result being that the step execution result can meet the data analysis requirements.

[0101] In an embodiment of the present application, the above-mentioned device also includes: a re-execution module, which is used to respond to the step evaluation result that the step execution result corresponding to the task step cannot meet the data analysis requirements, re-select a new external tool corresponding to the task step from the external tool library based on the step identifier of the task step and the data characteristics of the data to be analyzed, and use the re-selected new external tool to execute the task step to obtain a new step execution result corresponding to the task step, until the new external tool corresponding to the task step can meet the data analysis requirements.

[0102] It should be noted that the text acquisition module 502, task construction module 504, and task execution module 506 correspond to steps S202 to S206 in the above embodiment. The examples and application scenarios implemented by the three modules and the corresponding steps are the same, but are not limited to the contents disclosed in the above embodiment. It should be noted that the above modules or units can be hardware components or software components stored in a memory and processed by one or more processors. The above modules can also be part of the device and can be run in the client device 10 provided in the above embodiment.

[0103] According to an embodiment of the present application, another data analysis device for implementing the above data analysis method is also provided. Figure 6 is a structural block diagram of another data analysis device according to an embodiment of the present application, such as Figure 6 As shown, the device includes: a first display module 602 and a second display module 604 .

[0104] Among them, the first display module 602 is used to respond to input instructions on the operation interface and display task description text on the operation interface, wherein the task description text is used to reflect the data analysis requirements of the data to be analyzed; the second display module 604 is used to respond to processing instructions on the operation interface and display data analysis results on the operation interface, wherein the data analysis results are used to characterize the results obtained by using the data analysis system to perform data analysis tasks, and the data analysis task includes multiple task steps, and the multiple task steps are used to perform data analysis on the data to be analyzed according to the data analysis requirements. The data analysis task is constructed by the data analysis system by analyzing the task description text.

[0105] It should be noted that the first display module 602 and the second display module 604 correspond to steps S402 to S404 in the above embodiment. The examples and application scenarios implemented by the two modules and the corresponding steps are the same, but are not limited to the contents disclosed in the above embodiment. It should be noted that the above modules or units can be hardware components or software components stored in a memory and processed by one or more processors. The above modules can also be part of an apparatus and can be run in the client device 10 provided in the above embodiment.

[0106] According to another aspect of the embodiment of the present application, a data analysis system is also provided. Figure 7 is a structural block diagram of a data analysis system according to an embodiment of the present application. Figure 7 As shown, the device includes: a first process module 702 and a second process module 704.

[0107] The first process module 702 is used to execute any one of the above methods; the second process module 704 is connected to the first process module through a preset mechanism, and is used to render and output the data analysis results of the data to be analyzed.

[0108] In an optional solution of this embodiment, in order to facilitate user operation, a data analysis system is proposed in this application, and the user can directly perform corresponding data analysis operations in the data analysis system. In order to ensure the accuracy of the data analysis process, the data analysis system can include at least the above-mentioned first process module and second process module, wherein the first process module can be used to execute the aforementioned data analysis method, and the second process module can be used to render and output the obtained data analysis results, so as to facilitate user understanding. The first process module and the second process module can be connected through a preset mechanism, such as Electron's IPC mechanism (Inter-Process Communication, inter-process mechanism), shared memory, message queue, etc.

[0109] In an embodiment of the present application, the first process module includes: a communication client, connected to the communication service manager, used to send data analysis requests to the communication service manager, and receive response results corresponding to the data analysis requests returned by the communication service manager; a communication service manager, used to manage and schedule external tools corresponding to data analysis requests; a state management module, connected to the communication service manager, used to maintain data consistency of the data analysis system in different application scenarios; a model adapter, connected to the communication service manager, used to call the data analysis model corresponding to the data analysis request.

[0110] In an optional scheme of this embodiment, to facilitate the execution of the aforementioned data analysis method, the first process module may include a communication client, a communication service manager, a state management module and a model adapter, wherein the communication client can be connected to the communication service manager, can send data analysis requests to the communication service manager, and receive response results corresponding to the data analysis requests returned by the communication service manager; the communication service manager can be used to manage and schedule external tools corresponding to the data analysis requests; the state management module is connected to the communication service manager and can be used to maintain data consistency of the data analysis system in different application scenarios; the model adapter is connected to the communication service manager and can be used to call the data analysis model corresponding to the data analysis request.

[0111] In an embodiment of the present application, the second process module includes: a page layer for displaying data analysis results to users; a component layer for providing users with data processing functions required during the data analysis process; and a tool layer for monitoring the system performance of the data analysis system and the user behavior of users.

[0112] In an optional solution of this embodiment, in order to facilitate users to understand and view the data analysis results, the second process module may include the above-mentioned page layer and component layer, as well as the corresponding tool layer, wherein the page layer can be used to display the data analysis results to the user; the component layer can be used to provide users with data processing functions required in the data analysis process; the tool layer can be used to monitor the system performance of the data analysis system and the user behavior of the user.

[0113] In an embodiment of the present application, the page layer includes: a chat analysis page for receiving task description text and outputting data analysis results; a script page for generating code of a preset code type based on the data analysis requirements corresponding to the task description text, and building a corresponding target external tool based on the code; a tool configuration page for managing the target external tool based on tool adjustment instructions.

[0114] In an optional scheme of this embodiment, the page layer may include the above-mentioned chat analysis page, script page and tool configuration page, wherein the chat analysis page can be used to receive task description text and output data analysis results; the script page can be used to generate code of a preset code type based on the data analysis requirements corresponding to the task description text, and build the corresponding target external tool based on the code; the tool configuration page can be used to manage the target external tool based on the tool adjustment instructions.

[0115] To facilitate understanding of the data analysis system of this application, Figure 8This is a structural block diagram of a data analysis system according to an embodiment of the present application, wherein 702 represents the first process module, 704 represents the second process module, and the first process module and the second process module can communicate through the IPC mechanism. 802 represents the system's infrastructure, which may include but is not limited to: LLM SDK and MCP SDK, and the React+TypeScript technology stack for building the front-end interface. 804 represents the tool management module, which can be used to manage external tools.

[0116] Figure 9 is a schematic diagram of a chat analysis page according to an embodiment of the present application, such as Figure 9 As shown, the chat analysis page may include: a first quick tool section 902, an information display section 904, and a tool audit section 906. The first quick tool section may be configured with a variety of quick tools, such as a new session window, a knowledge base, bookmarks, a quick entry to the tool configuration page, a help tab, etc. These quick tools can reduce the difficulty of user data analysis. To facilitate user selection of appropriate quick tools, corresponding summary content may be set for different quick tools. Users may quickly obtain the summary content by touching or hovering the pointer over the quick tool icon to understand the currently selected quick tool. The information display section can be used to display conversation content that may be involved in the data analysis process, such as the task description text entered by the user, the data analysis results of the data to be analyzed, etc. To facilitate user operation, the right side can be set to display the task description text entered by the user, and the left side can be set to display the data analysis results obtained by the data analysis system. At the same time, an information input bar and some common tools can be set below the information display section, such as a model selector, shortcut phrases, knowledge base, model token limit prompt, model temperature parameter setter, model message quantity limit setter, etc., to facilitate user data analysis. The tool audit section can be used to display the usage of external tools used in the data analysis process, such as the name of the external tool, security verification results, etc. At the same time, to facilitate users to view the external tools used, the code of the external tool used can be displayed to the user in detail in the tool audit section. It can be the code of an existing external tool or the code generated in real time to execute the task steps.

[0117] Figure 10 is a schematic diagram of a tool configuration page according to an embodiment of the present application, such as Figure 10As shown, the tool configuration page may include at least: a second quick tool section 1002, a plug-in display section 1004, a plug-in control section 1006, and a plug-in market 1008. The second quick tool section may be similar to the first quick tool section described above and may also be configured with multiple quick tools, but the specific types of quick tools may be different from those in the first quick tool section and may be set by the user, which is not limited here. The plug-in display section may be used to display the external tools currently configured in the data analysis system to the user and may at least include the names of the configured external tools for easy viewing by the user. The plug-in control section may be used to control whether to enable the configured external tools and may be displayed to the user through buttons, options, etc. The corresponding user may click on the button or option to determine whether to use the configured external tool. The plug-in market may include multiple external tools that can be used in the data analysis system. The user may view these external tools by clicking on the corresponding option in the plug-in market, configure these external tools to the data analysis system, or remove external tools currently configured in the data analysis system.

[0118] Taking into account that the plug-in market may not contain the external tools required by users, that is, it may not be possible to execute a certain task step in the data analysis task through the external tools in the plug-in market, it is also possible to set up additional corresponding plug-in creation options in the above-mentioned tool configuration page. Users can use the plug-in creation option to open the corresponding plug-in creation area in the data analysis system to configure the corresponding external tools by themselves, thereby ensuring the execution effect of the data analysis task. Figure 11 This is a schematic diagram of a plug-in newly created area according to an embodiment of the present application. Figure 11 As shown, the new plug-in area may include: plug-in identification 1102, plug-in type 1104, plug-in description 1106, plug-in command 1108, environment variables 1110, plug-in code 1112 and other parts. Users can use these parts to configure external tools that need to be inserted into the data analysis system, so that the data analysis system can smoothly perform data analysis tasks based on the configured external tools.

[0119] It should be noted that the preferred implementation scheme involved in the above embodiments of the present application is the same as the scheme provided in the above embodiments, as well as the application scenario and implementation process, but is not limited to the scheme provided in the above embodiments.

[0120] An embodiment of the present application may provide a computing device. Figure 12 This is a structural block diagram of a computing device according to an embodiment of the present application. Figure 12 As shown, the computing device 1200 may include: one or more (only one is shown in the figure) processors 1202, a memory 1204, a storage controller, and a peripheral interface.

[0121] The above-mentioned computing device can be understood as an integrated intelligent terminal, including but not limited to a server, a desktop computer, a PC (Personal Computer), a model all-in-one machine, etc., and the computing device can be pre-installed with the model in the above-mentioned embodiment of this application.

[0122] Specifically, the computing device can pre-set multiple types of models, including but not limited to models in the fields of natural language processing, visual processing, speech processing, code processing, multimodal task processing, etc., so as to provide a variety of model choices. In different product forms, the computing device can support one or more model usage methods, including but not limited to model training, model calling, model fine-tuning, model deployment, model reasoning and application, etc. In some product forms, the computing device also supports model management, including but not limited to multi-type model management (supporting the management of multiple types of models such as discriminants and genesis), model version control (supporting the control of different model versions), model evaluation (based on model evaluation tools to evaluate the performance and effect of the model), etc. In other product forms, the computing device can also create applications based on the model, provide API calling capabilities, and can call the model to the created application through the API interface. At the same time, it provides application management tools to achieve application management and control.

[0123] Furthermore, the computing device can also include data management (supporting the creation and management of model tuning data sets), a training center (providing rich training resources to help users learn and master AI technology), and basic management and control capabilities (providing enterprise-level basic management and control capabilities to ensure the security and efficient operation of the system). Through the above functions, a comprehensive, integrated AI development, training, deployment and application device is provided.

[0124] Among them, the memory can be used to store software programs and modules, such as program instructions / modules corresponding to the methods and devices in the embodiments of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, implementing the methods in the above embodiments. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely located relative to the processor, and these remote memories may be connected to terminal A via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0125] The processor may call the executable program stored in the memory through the transmission device to execute the method of any one of the above embodiments.

[0126] An embodiment of the present application may provide an electronic device. Figure 13 This is a structural block diagram of an electronic device according to an embodiment of the present application. Figure 13 As shown, the electronic device may include: an input / output device 1302 ; a memory 1304 ; and a processor 1306 , wherein the processor 1306 is connected to the input / output device 1302 and the memory 1304 via a bus 1308 .

[0127] Among them, the memory can be used to store software programs and modules, such as program instructions / modules corresponding to the methods and devices in the embodiments of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, implementing the methods in the above embodiments. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely located relative to the processor, and these remote memories may be connected to terminal A via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0128] The processor may call the executable program stored in the memory through the transmission device to execute the method of any one of the above embodiments.

[0129] Those skilled in the art will understand that Figure 13 The structure shown is for illustration only, and the electronic device may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (Mobile Internet Devices, MID), a PAD, and other terminal devices. Figure 13 It does not limit the structure of the above electronic device. For example, the electronic device may also include Figure 13 More or fewer components (such as network interfaces, display devices, etc.) shown in the figure, or having Figure 13 Different configurations shown.

[0130] A person skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0131] The embodiment of the present application further provides a computer-readable storage medium. Optionally, in this embodiment, the computer-readable storage medium can be used to store the program code executed by the method provided in the above embodiment.

[0132] Optionally, in this embodiment, the above-mentioned storage medium may be located in or within the electronic device.

[0133] Optionally, in this embodiment, the computer-readable storage medium is configured to store an executable program, and when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the methods in the above embodiments.

[0134] The embodiment of the present application further provides a computer program product. Optionally, in this embodiment, the computer program product may include a computer program, and when the computer program is executed by a processor, the method provided in the embodiment is implemented.

[0135] The embodiments of the present application further provide a computer program product. Optionally, the computer program product may include a non-volatile computer-readable storage medium, which may be used to store a computer program that, when executed by a processor, implements the method provided in the embodiments above.

[0136] The embodiment of the present application further provides a computer program. Optionally, in this embodiment, when the computer program is executed by a processor, the method provided in the above embodiment is implemented.

[0137] In the above embodiments of the present application, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0138] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0139] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0140] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0141] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program code.

[0142] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A data analysis method, characterized in that: Applied to the client, including: Obtaining a task description text, wherein the task description text is used to reflect data analysis requirements for the data to be analyzed; Analyzing the task description text using a data analysis system to construct a data analysis task corresponding to the task description text, wherein the data analysis task includes a plurality of task steps, and the plurality of task steps are used to perform data analysis on the data to be analyzed according to the data analysis requirements; The data analysis system is used to execute the multiple task steps to obtain data analysis results of the data to be analyzed.

2. The method according to claim 1, characterized in that The data analysis system is configured with a data analysis model; the data analysis system is used to analyze the task description text to construct a data analysis task corresponding to the task description text, including: Performing semantic extraction on the task description text to obtain text semantic features of the task description text, wherein the text semantic features are used to reflect the data analysis requirements; Determining data source information and application scenarios corresponding to the data to be analyzed based on the text semantic features, wherein the data source information is used to reflect the validity of at least one data source related to the task description text; The text semantic features, the data source information and the application scenario are input into the data analysis model, and the data analysis task is generated using the data analysis model.

3. The method according to claim 2, characterized in that The data analysis model includes: a data source determination module, a knowledge base determination module, an external tool determination module, and an analysis task generation module; inputting the text semantic features, the data source information, and the application scenario into the data analysis model, and generating the data analysis task using the data analysis model, including: Based on the data source information, determine a target data source from the at least one data source using the data source determination module, wherein the target data source includes the data to be analyzed; Based on the application scenario, determining a target knowledge base from at least one knowledge base using the knowledge base determination module, wherein the target knowledge base includes data meanings and data usage rules corresponding to the data to be analyzed in the application scenario; Based on the text semantic features and the data to be analyzed, selecting at least one second external tool from a plurality of first external tools using the external tool determination module, and building an external tool library based on the at least one second external tool, wherein the at least one second external tool is used to represent an external tool capable of performing data processing on the data to be analyzed according to the data analysis requirements; The data source information corresponding to the target data source, the target knowledge base and the external tool library are input into the analysis task generation module, and the data analysis task is generated using the analysis task generation module.

4. The method according to claim 1, wherein The step of using the data analysis system to execute the plurality of task steps to obtain data analysis results of the data to be analyzed includes: Based on the task step, selecting a target external tool corresponding to the task step from an external tool library, wherein different task steps correspond to different target external tools, and the target external tool is used to perform data processing on the data to be analyzed in the corresponding task step; Executing the task step based on the target external tool to obtain a step execution result corresponding to the task step; In response to the completion of the execution of the multiple task steps, step execution results corresponding to the multiple task steps are summarized to obtain data analysis results of the data to be analyzed.

5. The method according to claim 4, characterized in that The selecting, based on the task step, a target external tool corresponding to the task step from an external tool library includes: Obtaining step identifiers of the task steps and data features of the data to be analyzed; Based on the step identifier, selecting at least one third external tool from the external tool library, wherein the at least one third external tool is used to represent an external tool capable of executing the task step; Evaluating the at least one third external tool based on the data features to obtain a tool evaluation result, wherein the tool evaluation result is used to reflect the correlation between the at least one third external tool and the data to be analyzed; Based on the tool evaluation result, a target external tool corresponding to the task step is selected from the at least one third external tool.

6. The method according to claim 4, characterized in that The step of executing the task step based on the target external tool to obtain a step execution result corresponding to the task step includes: Obtaining a structured query statement corresponding to the target external tool; Detecting query operations that can be executed by the structured query statement to obtain a statement detection result, wherein the statement detection result is used to indicate whether the query operation is safe; In response to the statement detection result indicating that the query operation is safe, the task step is executed based on the target external tool to obtain a step execution result corresponding to the task step.

7. The method according to claim 6, characterized in that The method further comprises: In response to the statement detection result indicating that the query operation is unsafe, based on the step identifier of the task step and the data characteristics of the data to be analyzed, a new external tool corresponding to the task step is re-selected from the external tool library, and if the re-selected new external tool is safe, the task step is executed based on the new external tool to obtain a step execution result corresponding to the task step.

8. The method according to claim 6 or 7, characterized in that The step of executing the task step based on the target external tool to obtain a step execution result corresponding to the task step includes: Detecting the code used by the target external tool to determine the code type of the code; In response to the code type being a preset code type, the target external tool is sent to a preset execution environment, and in the preset execution environment, the task step is executed based on the target external tool to obtain a step execution result corresponding to the task step.

9. The method according to claim 8, characterized in that After executing the task step based on the target external tool and obtaining a step execution result corresponding to the task step, the method includes: Performing format conversion on the step execution result corresponding to the task step to obtain a conversion execution result; Inputting the task description text and the conversion execution result into a data analysis model, and using the data analysis model to evaluate the step execution result to obtain a step evaluation result, wherein the step evaluation result is used to determine whether the step execution result corresponding to the task step can meet the data analysis requirements; In response to the step evaluation result being that the step execution result can meet the data analysis requirement, it is determined that the task step execution is completed.

10. The method according to claim 9, characterized in that The method further comprises: In response to the step evaluation result being that the step execution result corresponding to the task step cannot meet the data analysis requirements, based on the step identifier of the task step and the data characteristics of the data to be analyzed, a new external tool corresponding to the task step is re-selected from the external tool library, and the task step is executed using the re-selected new external tool to obtain a new step execution result corresponding to the task step, until the new external tool corresponding to the task step can meet the data analysis requirements.

11. A data analysis system, characterized in that: include: A first process module, configured to execute the method according to any one of claims 1 to 10; The second process module is connected to the first process module through a preset mechanism, and is used to render and output the data analysis results of the data to be analyzed.

12. The system according to claim 11, wherein: The first process module includes: A communication client, connected to the communication service manager, configured to send a data analysis request to the communication service manager and receive a response result corresponding to the data analysis request returned by the communication service manager; The communication service manager is used to manage and schedule external tools corresponding to the data analysis request; A state management module, connected to the communication service manager, for maintaining data consistency of the data analysis system in different application scenarios; A model adapter is connected to the communication service manager and is used to call a data analysis model corresponding to the data analysis request.

13. The system according to claim 11, wherein: The second process module includes: The page layer is used to display the data analysis results to the user; The component layer is used to provide the user with data processing functions required in the data analysis process; The tool layer is used to monitor the system performance of the data analysis system and the user behavior of the user.

14. The system according to claim 13, wherein: The page layer includes: The chat analysis page is used to receive the task description text and output the data analysis results; A script page is used to generate code of a preset code type based on the data analysis requirements corresponding to the task description text, and to build a corresponding target external tool based on the code; The tool configuration page is used to manage the target external tool based on the tool adjustment instruction.

15. A data analysis method, characterized in that: include: In response to an input instruction on the operation interface, displaying a task description text on the operation interface, wherein the task description text is used to reflect data analysis requirements for the data to be analyzed; In response to a processing instruction acting on the operation interface, a data analysis result is displayed on the operation interface, wherein the data analysis result is used to characterize the result obtained by using the data analysis system to perform a data analysis task, the data analysis task includes multiple task steps, and the multiple task steps are used to perform data analysis on the data to be analyzed according to the data analysis requirements. The data analysis task is obtained by the data analysis system analyzing the task description text and constructing it.

16. An electronic device, characterized in that: include: a memory storing an executable program; A processor is connected to the memory via a bus, and is configured to run the program, wherein the program executes the method according to any one of claims 1 to 10 when running.

17. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored executable program, wherein when the executable program is run, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 10.

18. A computer program product, characterized in that The invention comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 10.

Images