Data analysis method and device, electronic equipment, and storage medium

Abstract

This application discloses a data analysis method, apparatus, electronic device, and storage medium, relating to the field of big data technology or other related fields. The method includes: receiving a data query request and data analysis requirements input by a target user in natural language through an interactive interface; parsing the data query request and data analysis requirements to obtain query configuration indicators and analysis configuration indicators; querying a preset database based on the query configuration indicators to obtain data query results, wherein the preset database is updated in a data background at a preset frequency; and analyzing the data query results based on the analysis configuration indicators to obtain data analysis results. This application solves the technical problem of cumbersome enterprise data analysis processes leading to low data analysis efficiency in related technologies.

Description

Technical Field

[0001] This invention relates to the field of big data technology or other related fields, and more specifically, to a data analysis method and apparatus, electronic device, and storage medium. Background Technology

[0002] With the rapid development of technologies such as big data, cloud computing, and artificial intelligence, corporate decision-making increasingly relies on in-depth analysis and insights into massive amounts of data. This is especially true in the financial industry, where the real-time nature and accuracy of data are crucial for risk management and customer experience optimization. However, traditional data analysis methods are often limited by complex manual configuration processes, requiring users to meticulously configure database information, metadata, and complex SQL statements. This cumbersome process is not only time-consuming and labor-intensive but also involves high learning costs and potential human error risks, severely restricting the efficiency and agility of data analysis.

[0003] Even more challenging is that when faced with ever-changing market demands, businesses often need to conduct real-time personalized data analysis to optimize internal processes. However, existing data analysis platforms generally lack flexible natural language processing capabilities and cannot directly understand and execute data requirements in unstructured text, making it difficult for businesses to respond quickly and analyze efficiently.

[0004] Taking software development efficiency evaluation in the financial industry as an example, R&D teams need to regularly monitor multiple key indicators, including code quality, project progress, and defect density, to ensure high-quality and timely delivery of software products. However, current data analysis tools often require technical teams to manually configure data sources and construct SQL query statements when handling such needs, and then analysts need to analyze and interpret the data one by one. This process is both complex and inefficient, and it is difficult to adapt to the fast-paced R&D environment.

[0005] In summary, the cumbersome data analysis process, lack of natural language understanding and execution capabilities, and difficulty in conducting personalized data analysis in related technologies have become major bottlenecks hindering enterprises from improving data processing efficiency and responding to market changes.

[0006] There is currently no effective solution to the above problems. Summary of the Invention

[0007] The main objective of this application is to provide a data analysis method, apparatus, electronic device, and storage medium to at least solve the technical problem in the related art that the data analysis process for enterprises is cumbersome, resulting in low efficiency in data analysis.

[0008] To achieve the above objectives, according to one aspect of this application, a data analysis method is provided. The method includes: receiving a data query request and data analysis requirements input by a target user in natural language via an interactive interface; parsing the data query request and the data analysis requirements to obtain query configuration indicators and analysis configuration indicators; querying a preset database based on the query configuration indicators to obtain data query results, wherein the preset database is updated in a data background at a preset frequency; and analyzing the data query results based on the analysis configuration indicators to obtain data analysis results.

[0009] Further, the step of parsing the data query request and the data analysis requirement to obtain query configuration indicators and analysis configuration indicators includes: calling the input interface of the target agent, wherein the target agent is pre-connected to a domain knowledge base and a deep learning model, and the target agent is used to understand natural language and convert it into computer language based on the domain knowledge base and the deep learning model; sending the data query request and the data analysis request to the input interface, and generating the query configuration indicators and the analysis configuration indicators through the target agent.

[0010] Furthermore, the step of generating the query configuration metrics and the analysis configuration metrics through the target proxy includes: performing semantic parsing on the data query request and the data analysis requirement through the deep learning model to obtain query intent keywords and analysis target keywords; retrieving the domain knowledge base based on the query intent keywords to obtain the query configuration metrics that match the query intent; and retrieving the domain knowledge base based on the analysis target keywords to obtain the analysis configuration metrics that match the analysis target.

[0011] Furthermore, the domain knowledge base is constructed through the following steps: obtaining the latest updated architecture information of the preset database from the data backend, wherein the architecture information includes: database information, table information, and field information; constructing a metadata index of the preset database based on the architecture information, and determining Q indicator entries based on the metadata index, wherein Q is a positive integer; and integrating all the indicator entries to obtain the domain knowledge base.

[0012] Furthermore, the update step of the preset database includes: collecting data from N data sources based on the preset frequency to obtain data to be updated, where N is a positive integer; preprocessing the data to be updated, wherein the preprocessing includes the following operations: data cleaning and format conversion; performing data consistency checks on the data to be updated from different data sources, and updating the preset database using the data to be updated if the data is consistent.

[0013] Further, the step of querying a preset database based on the query configuration indicators to obtain data query results includes: constructing a database query statement based on the query configuration indicators and sending the database query statement to the preset database; executing the database query statement in the preset database and receiving the data query results returned by the preset database.

[0014] Further, the step of analyzing the data query results based on the analysis configuration indicators to obtain data analysis results includes: determining the analysis objectives and analysis process based on the analysis configuration indicators, wherein the analysis process includes at least: data feature insight, data trend analysis, and data anomaly detection; constructing a data analysis workflow based on the analysis objectives and the analysis process; inputting the data query results into the data analysis workflow and outputting an analysis report; and displaying the analysis report on the interactive interface to obtain the data analysis results.

[0015] To achieve the above objectives, according to another aspect of this application, a data analysis apparatus is also provided, comprising: a receiving unit for receiving a data query request and data analysis requirements input by a target user in natural language form through an interactive interface; a parsing unit for parsing the data query request and the data analysis requirements to obtain query configuration indicators and analysis configuration indicators; a query unit for querying a preset database according to the query configuration indicators to obtain data query results, wherein the preset database is updated in a data background at a preset frequency; and an analysis unit for analyzing the data query results according to the analysis configuration indicators to obtain data analysis results.

[0016] Furthermore, the parsing unit includes: a retrieval module, used to retrieve the input interface of the target agent, wherein the target agent is pre-connected to a domain knowledge base and a deep learning model, and the target agent is used to understand natural language and convert it into computer language based on the domain knowledge base and the deep learning model; and a generation module, used to send the data query request and the data analysis request to the input interface, and generate the query configuration indicators and the analysis configuration indicators through the target agent.

[0017] Furthermore, the generation module includes: a semantic parsing submodule, used to perform semantic parsing on the data query request and the data analysis requirements through the deep learning model to obtain query intent keywords and analysis target keywords; a first retrieval submodule, used to retrieve the domain knowledge base based on the query intent keywords to obtain the query configuration indicators matching the query intent; and a second retrieval submodule, used to retrieve the domain knowledge base based on the analysis target keywords to obtain the analysis configuration indicators matching the analysis target.

[0018] Furthermore, the data analysis device further includes: an acquisition module, used to acquire the latest updated architecture information of the preset database from the data backend, wherein the architecture information includes: database information, table information, and field information; a first construction module, used to construct a metadata index of the preset database based on the architecture information, and determine Q indicator entries based on the metadata index, wherein Q is a positive integer; and an integration module, used to integrate all the indicator entries to obtain the domain knowledge base.

[0019] Furthermore, the data analysis device further includes: a data acquisition module, used to acquire data from N data sources based on the preset frequency to obtain data to be updated, where N is a positive integer; a preprocessing module, used to preprocess the data to be updated, wherein the preprocessing includes the following operations: data cleaning and format conversion; and an update module, used to perform data consistency checks on the data to be updated from different data sources, and, if the data is consistent, use the data to be updated to update the preset database.

[0020] Furthermore, the query unit includes: a sending module, configured to construct a database query statement based on the query configuration indicators and send the database query statement to the preset database; and an execution module, configured to execute the database query statement in the preset database and receive the data query result returned by the preset database.

[0021] Furthermore, the analysis unit includes: a determination module, used to determine the analysis objective and analysis process based on the analysis configuration indicators, wherein the analysis process includes at least: data feature insight, data trend analysis, and data anomaly detection; a second construction module, used to construct a data analysis workflow based on the analysis objective and the analysis process; an input module, used to input the data query results into the data analysis workflow and output an analysis report; and a feedback module, used to feed back the analysis report to the interactive interface for display, thereby obtaining the data analysis results.

[0022] To achieve the above objectives, according to another aspect of this application, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the data analysis method described in any one of the above claims.

[0023] To achieve the above objectives, according to another aspect of this application, an electronic device is also provided, including one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the data analysis method described in any of the above claims.

[0024] To achieve the above objectives, according to another aspect of this application, a computer program product is also provided, including computer instructions, wherein when the computer instructions are executed by a processor, they implement the steps of the data analysis method described in any one of the above claims.

[0025] This invention proposes a data analysis method. First, it receives data query requests and data analysis requirements input by target users in natural language through an interactive interface. Then, it parses the data query requests and data analysis requirements to obtain query configuration indicators and analysis configuration indicators. Next, it queries a preset database based on the query configuration indicators to obtain data query results. The preset database is updated in the data background at a preset frequency. Finally, it analyzes the data query results based on the analysis configuration indicators to obtain data analysis results.

[0026] This invention employs a combination of intelligent agents and natural language processing. By constructing a dynamic domain knowledge base and using deep learning models, it achieves the goal of automating the parsing and execution of user natural language data query and analysis requests. This significantly improves the agility and efficiency of enterprise data analysis. Specifically, this invention utilizes a pre-connected domain knowledge base and deep learning technology in the intelligent agent to accurately understand and transform user data requests expressed in everyday spoken language, generating precise query and analysis configuration indicators. This avoids the tedious manual configuration process of traditional methods. Simultaneously, the pre-set database ensures data timeliness and consistency through regular automatic updates. More importantly, this invention uses deep learning models to semantically understand user query and analysis requests, achieving efficient conversion of natural language into database query statements. It can also adjust analysis strategies based on analysis configuration indicators and perform deep analysis feedback to users, greatly simplifying the data analysis process and improving analysis accuracy. Through these steps, it not only significantly reduces the workload of data analysts but also enables enterprises to make more timely and accurate decisions, thereby solving the technical problem of cumbersome enterprise data analysis processes leading to low data analysis efficiency in related technologies. Attached Figure Description

[0027] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0028] Figure 1 A hardware structure block diagram of a computer terminal (or mobile device) for implementing data analysis methods is shown.

[0029] Figure 2 This is a flowchart of an optional data analysis method according to an embodiment of the present invention;

[0030] Figure 3 This is a schematic diagram of an optional data analysis device according to an embodiment of the present invention;

[0031] Figure 4 This is a structural block diagram of an electronic device for performing a data analysis method according to an embodiment of the present invention. Detailed Implementation

[0032] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0033] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0034] To facilitate understanding of the present invention by those skilled in the art, some terms or nouns involved in the various embodiments of the present invention are explained below:

[0035] Agent, or intelligent agent, in this invention refers to a software system with artificial intelligence (AI) capabilities that can understand and parse users' natural language requests and generate corresponding computer instructions, such as SQL query statements or analysis strategies.

[0036] A domain knowledge base is a database containing information about a specific domain, such as the domain of R&D performance measurement involved in this invention. The knowledge base not only stores information about the database architecture (such as database names, table names, and field names), but also includes natural language descriptions of these elements, as well as predefined analysis methods and metrics, helping intelligent agents better understand and execute user queries.

[0037] NLP, or Natural Language Processing, is a branch of artificial intelligence that focuses on enabling computers to understand, parse, and generate human natural language (such as English and Chinese). In this invention, it is used to analyze user-input natural language requests, extract query intent keywords and analysis target keywords, thereby generating specific query and analysis configuration metrics.

[0038] Large Language Models (MLMs) are deep learning models with a large number of parameters, trained on massive datasets, and capable of handling complex natural language tasks. In this invention, large MLMs are used to improve the accuracy of intelligent agents in understanding and generating query statements.

[0039] Deep learning is a subset of machine learning techniques that uses multi-layered neural networks to learn the complex structures and patterns of data. In this invention, a deep learning model is used at the core of an intelligent agent to improve its ability to understand natural language and generate configuration metrics.

[0040] It should be noted that the data analysis method and apparatus in this application can be used in the field of big data technology for intelligent analysis and personalized insights into enterprise software development efficiency indicators, and can also be used in any field other than big data technology for intelligent analysis and personalized insights into enterprise software development efficiency indicators. This application does not limit the application field of the data analysis method and apparatus.

[0041] It should be noted that all relevant information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, and displayed data) involved in this application are information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, processing, transmission, provision, disclosure, use, and handling of such data comply with the laws, regulations, and standards of the relevant regions, necessary confidentiality measures have been taken, and they do not violate public order and good morals. Corresponding access points are provided for users to choose to authorize or refuse access. For example, this system has interfaces with relevant users or organizations. Before obtaining relevant information, a request to obtain the information needs to be sent to the aforementioned user or organization through the interface, and the relevant information is obtained only after receiving consent from the aforementioned user or organization.

[0042] The information collection (e.g., user voice, video, and text collection) and analysis operations involved in this application have provided users with corresponding operation entry points during execution, allowing users to choose to agree to or reject the automated decision results; if the user chooses to reject, the process will proceed to the expert decision-making process.

[0043] The following embodiments of the present invention can be applied to various systems / applications / devices that require enterprise-level data querying and intelligent data analysis, enabling conversational data insights and performance enhancement based on deep learning. The present invention uses intelligent agents and natural language processing technology to understand and convert user requests, and then automatically executes database queries and data analysis based on the parsing results. This simplifies the data query process, enhances the personalization and intelligence of data analysis, and provides rapid response and accurate results for complex data needs and analysis scenarios.

[0044] The present invention will now be described in detail with reference to various embodiments.

[0045] Example 1

[0046] According to an embodiment of the present invention, an embodiment of a data analysis method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0047] The data analysis method embodiment provided in Embodiment 1 of the present invention can be executed on a mobile terminal, computer terminal or similar computing device. Figure 1 A hardware block diagram of a computer terminal (or mobile device) for implementing data analysis methods is shown. Figure 1As shown, the computer terminal 10 (or mobile device) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0048] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0049] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the data analysis method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the above-mentioned data analysis method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0050] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0051] The display may be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0052] Under the above operating environment, the present invention provides, as follows: Figure 2 The data analysis method shown is implemented by an intelligent data analysis system, which combines natural language processing and deep learning technologies. It is used for enterprise software development efficiency indicator analysis scenarios, especially to solve the problems of cumbersome data query configuration and personalized analysis needs. Through intelligent agents and dynamic knowledge base construction, the specific steps are: recognizing user natural language queries, intelligently generating query and analysis configuration indicators, automatically executing database queries, dynamically adjusting analysis strategies, and generating insight reports. This aims to simplify the data query process and improve the efficiency and personalization of data analysis.

[0053] First, the implementation system uses an intelligent agent to identify and understand the data query and analysis needs expressed by users in natural language. Second, based on the understanding results, the intelligent agent automatically generates query configuration indicators and simultaneously calls the backend data synchronization service to query the preset database and obtain the data query results. Next, based on the analysis configuration indicators, the implementation system uses a deep learning model to dynamically adjust the analysis strategy and gain in-depth insights into the query results. Finally, the implementation system integrates the analysis results with expert knowledge base information to form an intuitive and practical insight report, helping users quickly identify and solve problems and improve the overall efficiency of software development.

[0054] The embodiments of the present invention will now be described in detail with reference to each specific step.

[0055] Figure 2 This is a flowchart of an optional data analysis method according to an embodiment of the present invention, such as... Figure 2 As shown, the method includes the following steps:

[0056] Step S201: Receive data query requests and data analysis needs from the target user in natural language form through the interactive interface.

[0057] It should be noted that the interactive interface refers to a visual platform or interface through which the target user communicates with the implementation system. It allows users to input query and analysis requirements in natural language, supports diverse input methods (such as text input and voice recognition), and can effectively present complex data results. A data query request refers to a request from a target user to retrieve specific information from a database or data warehouse, and at least includes details such as the range of data to be queried, the time span, and the type of indicator. In this embodiment, the data query request can primarily focus on software development efficiency indicators, such as code submission frequency, test coverage, and defect density.

[0058] Unlike simple data queries, data analysis focuses on in-depth interpretation of retrieved data to uncover trends, patterns, or potential problems. In software development performance measurement scenarios, data analysis needs can include forecasting project progress, providing optimization suggestions for resource allocation, and investigating the causes of defects, thus providing data support for decision-making.

[0059] The above steps provide a convenient platform that allows financial users to accurately express their data query and analysis needs using only natural language, without needing to understand the underlying data architecture in detail. The system can then automatically understand and execute these requests, thereby lowering the threshold for data operations and improving the efficiency and flexibility of data analysis.

[0060] Step S202: Parse the data query request and data analysis requirements to obtain query configuration indicators and analysis configuration indicators.

[0061] It should be noted that query configuration metrics and analysis configuration metrics are specific executable parameter sets that the implementation system transforms after understanding the natural language request of the target user. These metrics are not limited to the dimensions, time range, and metric types of data queries, but can also include the data analysis methods, depth, data models and algorithms involved.

[0062] Specifically, query configuration metrics refer to a series of parameters generated by the implementation system based on the query needs of the target user, used to guide query operations in the database or data warehouse. For example, in the scenario of software development performance measurement, query configuration metrics may include project ID, code submission time range, developer ID, required metrics (such as lines of code, test case pass rate), etc., to ensure that the implementation system can accurately retrieve the data required by the user from the data source.

[0063] Analytical configuration metrics go a step further, encompassing the target user's needs for in-depth data interpretation and insight. This can involve the type of data analysis (trend analysis, comparative analysis, predictive analysis, etc.), data modeling methods (machine learning, statistical analysis), and visualization requirements (chart type, data display preferences). For example, if the target user wants to compare the code quality of two teams over different time periods, the analytical configuration metrics could include identifying the two teams, comparing the time ranges, defining code quality (such as defect rate), and visualization methods (line charts, heatmaps), etc.

[0064] The above steps transform users' vague, unstructured natural language requests into structured, explicit query and analysis instructions, thereby achieving more efficient and accurate data operations. This not only reduces the configuration difficulty for users due to the need to understand complex data structures, but also improves the system's response speed and the accuracy of data analysis.

[0065] Optionally, in the data analysis method provided in this embodiment of the invention, the step of parsing the data query request and data analysis requirements to obtain query configuration indicators and analysis configuration indicators includes: calling the input interface of the target agent, wherein the target agent is pre-connected to the domain knowledge base and deep learning model, and the target agent is used to understand natural language and convert it into computer language based on the domain knowledge base and deep learning model; sending the data query request and data analysis request to the input interface, and generating query configuration indicators and analysis configuration indicators through the target agent.

[0066] In embodiments of the present invention, when a target user submits a data query request and data analysis requirement through an interactive interface, the internal operation process of the system involves: invoking the target agent and the collaborative work between the target agent, the domain knowledge base, and the deep learning model. It should be noted that the target agent is an intelligent component integrating deep learning capabilities and domain knowledge, whose function is to parse the user's natural language input and convert it into query configuration indicators and analysis configuration indicators that the computer can understand.

[0067] In this embodiment of the invention, the target agent is pre-connected to a domain knowledge base and a deep learning model. The deep learning model can master the structure and patterns of natural language through a large amount of training data and thus understand the user's intent. The domain knowledge base contains terms, concepts and common query patterns of a specific domain, which helps the model map the user's natural language request to a specific domain operation.

[0068] In practice, the deep learning model of the target agent uses embedding technology to convert natural language into vector representations, which capture the meaning of words and their context. Subsequently, the deep learning model performs inference based on the vector representations, identifying key information in the user request, such as the query target, time range, and data analysis type. This key information is then transformed into structured query configuration metrics and analysis configuration metrics.

[0069] Alternatively, in this embodiment, the intelligent agent, in addition to basic parsing and conversion functions, can also implement two functions: a function of dynamically adjusting analysis strategies and a function of intelligent recommendation analysis methods. It should be noted that function one refers to dynamically adjusting the analysis configuration indicators based on changes in target user needs, such as changing the time window for trend analysis or adding additional comparison dimensions. Implementing function one requires configuring the target agent to support real-time strategy adjustment and model retraining mechanisms. Function two refers to recommending the most relevant analysis methods based on the target user's historical query and analysis behavior, improving analysis efficiency. Implementing function two requires configuring the target agent to record and learn user behavior patterns, which may involve the integration of reinforcement learning or personalized recommendation algorithms.

[0070] Another point to note is that, in this embodiment, the input interface acts as a receiver of user input, which can be a web form, API interface, or command-line interface. Its function is to act as a bridge between the user and the intelligent agent, receiving the user's natural language requests and forwarding them to the target agent for processing. The input interface design must ensure the convenience of user input and the accurate parsing of requests by the intelligent agent.

[0071] Compared with traditional data transmission methods (such as file transfer protocols and direct database queries), the advantages of input interface transmission are: 1. Higher security: Data is transmitted through encrypted API requests, reducing the risk of data leakage; 2. Better flexibility: Supports multiple data formats and transmission protocols, allowing users to choose the most suitable input method according to their needs; 3. Stronger intelligent processing capabilities: Directly converts user input into intelligent agent processing requests, rather than simple data transmission, improving the overall intelligence level of the system.

[0072] In one specific implementation of the input interface, the target user can submit data query requests and analysis needs in natural language through an interactive interface (such as a web application, mobile application, etc.). These requests are sent to the input interface of the implementation system backend via Hypertext Transfer Protocol / Secure Hypertext Transfer Protocol. Upon receiving the request, the input interface performs preliminary format validation and parsing, and then forwards it to the target agent for further processing. The target agent generates corresponding query configuration indicators and analysis configuration indicators based on its understanding of deep learning models and domain knowledge bases, and finally feeds them back to the implementation system to execute specific database queries and data analysis tasks.

[0073] The above steps construct a data analysis system based on intelligent agents, geared towards measuring software development performance. By automatically translating users' natural language requirements into precise query and analysis commands, it automates and intelligently processes data, simplifying data operation workflows, improving the efficiency and depth of data analysis, and providing software development teams with immediate and accurate performance insights, thus supporting project management and optimization. In this process, the core value of the intelligent agent lies in its powerful natural language understanding and conversion capabilities, as well as its domain-knowledge-based intelligent decision support, enabling users without technical backgrounds to easily perform complex data queries and analyses.

[0074] Optionally, in the data analysis method provided in this embodiment of the invention, the step of generating query configuration indicators and analysis configuration indicators through a target proxy includes: performing semantic parsing on the data query request and data analysis requirements using a deep learning model to obtain query intent keywords and analysis target keywords; retrieving a domain knowledge base based on the query intent keywords to obtain query configuration indicators that match the query intent; and retrieving a domain knowledge base based on the analysis target keywords to obtain analysis configuration indicators that match the analysis target.

[0075] It should be noted that the deep learning model is a pre-selected machine learning algorithm based on a neural network architecture in this embodiment of the invention. It is capable of handling the natural language processing tasks in this embodiment. In the data analysis method provided by this embodiment, the deep learning model extracts query intent keywords and analysis target keywords through semantic parsing of data query requests and data analysis requirements, thereby guiding query and analysis operations. This embodiment of the invention may use a deep learning model with a Transformer architecture.

[0076] Specifically, in a Transformer-based deep learning model, an embedding layer is first used to convert text into vector representations, serving as the foundation for the model's understanding and processing of natural language. Next, the self-attention mechanism in the deep learning model allows the model to consider the interdependencies between different parts of the text, facilitating the understanding of complex queries and analysis commands and obtaining internal model data. Finally, the decoder or classification head in the deep learning model is used to convert the internal data representation into the final keyword output. In the semantic parsing process, the principle is that the model first converts each word into a high-dimensional vector through the embedding layer, then captures the relationships between words in the text through the self-attention mechanism, and finally generates a set of keywords most likely expressing the user's intent based on contextual understanding.

[0077] Furthermore, query intent keywords can include specific metric names (such as "number of code commits"), time limits ("last week", "last three months"), and object identifiers ("Project A", "Developer B"), etc., to precisely pinpoint the specific requirements of the data query. Analysis target keywords can include analysis type ("trend analysis", "comparative analysis"), specific analysis metrics ("code quality", "resource utilization"), and analysis dimensions ("by module", "by developer"), etc., to guide the direction and depth of data analysis.

[0078] Another point to note is that the domain knowledge base contains detailed information on terminology, metric definitions, common query patterns, and data analysis methods related to software development efficiency measurement. The knowledge base content can be specifically presented as: definitions of development metrics, such as lines of code, test case pass rate, and defect fix time; common data query patterns, including SQL query templates and examples; descriptions of analysis methods, such as statistical analysis and the application of machine learning algorithms; and expert experience summaries, providing guidance on how to conduct effective analysis based on specific metrics. The purpose of retrieving the domain knowledge base is to transform user-provided keywords into specific query and analysis configuration metrics, ensuring that the system understands the user's intent and executes the correct data operations.

[0079] In determining whether query configuration metrics match, predefined query patterns in the domain knowledge base can be used as a basis. The model compares the user-provided query intent keywords with entries in the knowledge base to determine whether the keywords can be mapped to specific query operations. For example, "number of code submissions" should match the "code submissions" metric.

[0080] In determining whether the analysis configuration metrics match, the model can also be used to compare the analysis target keywords with the analysis methods and metric definitions in the domain knowledge base to confirm whether the keywords accurately reflect the user's analysis needs. For example, "code quality" analysis may need to be associated with metrics such as "defect density" or "test coverage".

[0081] Optionally, in the data analysis method provided in this embodiment of the invention, the domain knowledge base is constructed through the following steps: obtaining the architecture information of the latest updated preset database in the data backend, wherein the architecture information includes: database information, table information, and field information; constructing a meta-information index of the preset database based on the architecture information, and determining Q indicator entries based on the meta-information index, wherein Q is a positive integer; and integrating all indicator entries to obtain the domain knowledge base.

[0082] It should be noted that obtaining the architecture information of the preset database, including database information, table information, and field information, is the first step in building a domain knowledge base. Its purpose is to provide a foundation for the subsequent construction of metadata indexes, ensuring that the intelligent agent can accurately understand the structure and content of the database, thereby accurately parsing the user's natural language requests and generating corresponding query configuration metrics.

[0083] The implementation steps for building a metadata index include: Information extraction, which involves obtaining the architecture information of a pre-defined database, including the database name, table names and structures, field types and comments, etc., through the metadata query function of the database management system or by using database connection interfaces, intelligent agents, or dedicated construction scripts. Index building involves processing the extracted architecture information to build the metadata index, including steps such as data structuring, field standardization, and index optimization to ensure the efficiency of the index and the accuracy of queries. Storage and updating involve storing the built metadata index in an appropriate data structure, such as an inverted index, a B-tree (self-balancing multi-way search tree), or a hash table, and periodically updating the index synchronously based on the updates to the pre-defined database.

[0084] It's important to note that metadata indexes can be represented as a structured database containing detailed information about each table and field in the pre-defined database, such as field names, types, descriptions, and related query and analysis logic. In some cases, the index may also include information about relationships between fields, such as foreign key associations and the business meaning of fields. Metadata indexes are crucial for intelligent agents to parse user requests, enabling them to quickly locate and identify the metrics and fields mentioned in user queries, as well as their position and meaning within the pre-defined database. This effectively avoids query errors or analytical biases caused by inaccurate data understanding.

[0085] The steps for determining indicator entries include: Keyword mapping, where the intelligent agent, based on metadata indexing, maps query intent keywords and analysis target keywords extracted from user requests to specific indicators or fields in the database. Logical matching involves further identifying the implicit logical relationships in user requests, such as time range, comparison objects, and analysis types, to determine which indicator entries best match the user's needs. Entry integration involves the intelligent agent integrating all relevant indicator entries based on the above information to form a complete set of query configuration indicators and analysis configuration indicators.

[0086] It should be further explained that the metric entries are metadata descriptions of specific metrics or fields corresponding to the keywords in the user request, as well as configuration information required to execute queries or analyses. For example, for a query of "the number of code commits for project X in the most recent month", the metric entry may include: the identifier of project X (such as project ID); the definition of the number of code commits (such as lines of code, number of commits); the time range (such as the most recent month); and the query method (such as SQL statement template).

[0087] Step S203: Query the preset database according to the query configuration indicators to obtain the data query results. The preset database is updated in the data background at a preset frequency.

[0088] It should be noted that a pre-prepared database refers to a database in a financial system used to store and manage data. In the context of software development performance measurement, a pre-prepared database can be a wide-table database specifically designed to collect and integrate development metrics (such as code submissions, defect tracking, and test results) to ensure data comprehensiveness and consistency.

[0089] The pre-defined database is updated at a preset frequency in the data backend. The data backend refers to the back-end system responsible for data processing, storage, and maintenance, including modules for data source access, data cleaning, and data transformation. The data backend acts as a bridge connecting the front-end user interface and the underlying data. The data backend automatically updates the data in the pre-defined database according to preset time intervals (i.e., the preset frequency) to ensure data real-time performance and validity. For example, the preset frequency can be set to once a day to ensure that R&D performance indicators in the database are updated daily, reflecting the latest project status and team performance.

[0090] The data query results are data information retrieved from the preset database by the implementation system based on the query configuration indicators. This can include specific values ​​such as code submission frequency and test coverage, or more complex datasets, such as the amount of code modifications made by each developer in the past month, and a comparison of defect density in different modules.

[0091] The above steps ensure the timeliness and accuracy of data query results by regularly updating the preset database. At the same time, the automated processing capabilities of the data backend can reduce manual intervention and improve the efficiency and reliability of data processing.

[0092] Optionally, in the data analysis method provided in this embodiment of the invention, the step of updating the preset database includes: collecting data from N data sources based on a preset frequency to obtain data to be updated, where N is a positive integer; preprocessing the data to be updated, wherein the preprocessing includes the following operations: data cleaning and format conversion; performing data consistency checks on the data to be updated from different data sources, and updating the preset database using the data to be updated if the data is consistent.

[0093] It should be noted that data sources can include various types of data storage and generation systems, collectively forming a comprehensive data foundation for measuring software development performance. Specific data sources may include:

[0094] Code management system: The data collected mainly includes code commit records, code review information, code change statistics, etc., and is usually presented as event logs or database records of the version control system.

[0095] Defect tracking system: The collected data includes defect reports, repair status, defect density, etc., and this data usually exists in the form of structured database tables.

[0096] Continuous integration / continuous deployment system: The collected data includes build status, test coverage, deployment frequency, etc., and is stored in the form of system logs or metrics database.

[0097] Project management tools: The data collected includes project progress, task status, resource allocation, etc., and is usually obtained through the database or API interface of the project management system.

[0098] Furthermore, preprocessing of updated data is a crucial step in ensuring data quality, format consistency, and data uniformity. Besides data cleaning and format conversion, preprocessing can also include: Data deduplication: removing duplicate records to ensure data uniqueness. Data validation: checking whether the data conforms to predefined formats and logical rules, such as the correctness of date formats and the reasonableness of numerical ranges. Data encryption or anonymization: encrypting or anonymizing data containing sensitive information, such as personal identification information or key project data points, to protect data and user privacy.

[0099] The necessary preprocessing operations include at least data cleaning, format conversion, and data deduplication. These operations are directly related to the integrity and accuracy of the data and are essential conditions before data updates. Data cleaning aims to remove or correct erroneous, incomplete, or irrelevant data; format conversion ensures that all data conforms to the required format of the preset database; and data deduplication avoids duplicate data in the database and improves data query efficiency.

[0100] It's important to further clarify that data consistency checking refers to verifying the consistency of data from different data sources before updating the default database. This ensures that the update operation does not introduce data conflicts or inconsistencies. Specific implementation steps include: Data source comparison: Comparing data from different data sources for the same metrics or fields to check if they reflect consistent information. Timestamp verification: Using timestamps to ensure the chronological order of data, avoiding data chaos caused by asynchronous updates. Logical checks: Verifying the logical relationships between data, such as the correlation between code commits and lines of code, to ensure logical consistency.

[0101] Methods for determining data consistency can be based on predefined rules or algorithms, such as comparing timestamps, verifying logical relationships between data, or using checksum techniques to verify data consistency. Data consistency checks are essential because they affect the reliability of the pre-defined database and the accuracy of data queries.

[0102] The specific steps for updating using the data to be updated include: Data integration: Integrating the preprocessed and consistency-checked data into a unified update format. Update strategy selection: Selecting an appropriate update strategy based on the characteristics of the preset database, such as batch update or row-by-row update. Executing the update operation: Using preset SQL statements or other database update mechanisms, writing the data to be updated into the preset database. The execution result may be the addition, modification, or deletion of indicator data in the updated database, ensuring that the database content reflects the latest R&D performance information.

[0103] The steps outlined above enable the construction of an automated, intelligent data analytics framework. This framework can automatically collect and update data from multiple data sources, understand user intent through deep learning models and domain knowledge bases, generate query and analysis configuration metrics, and ultimately provide accurate R&D performance metrics and insights. This framework aims to simplify the complexity of R&D performance measurement, improve the efficiency and accuracy of data analysis, and support software development teams in making data-driven decisions, thereby optimizing R&D processes and improving project management efficiency.

[0104] Optionally, in the data analysis method provided in this embodiment of the invention, the step of querying a preset database based on query configuration indicators to obtain data query results includes: constructing a database query statement based on the query configuration indicators and sending the database query statement to the preset database; executing the database query statement in the preset database and receiving the data query results returned by the preset database.

[0105] It should be noted that the specific operation of constructing database query statements based on query configuration metrics can be as follows: The intelligent agent, based on the query configuration metrics extracted from the user request, including but not limited to metric names, time ranges, and object identifiers (such as teams, projects, or individuals), uses an SQL knowledge base to generate or automatically generate SQL query statements that conform to a preset database structure. During this process, the intelligent agent needs to ensure an accurate correspondence between the query statements and the metric entries, while also considering database query optimization rules to improve query efficiency.

[0106] Suppose a user requests "get the number of code commits for project Y last month", the query generated by the smart agent might look like this:

[0107] SQL

[0108] SELECT COUNT( )

[0109] FROM code_commits

[0110] WHERE project_id = 'Y'

[0111] AND DATE(commit_time) BETWEEN DATE_SUB(CURDATE(), INTERVAL 1 MONTH)AND CURDATE();

[0112] The above is a typical SQL query statement, which specifies that all records related to project Y should be retrieved from the code_commits table, and the commit_time field value of these records should fall within the time range of "last month". Finally, the number of query results is counted, which is the number of code commits.

[0113] Once the query statement is constructed and sent to the preset database via the input interface, the database can perform the following steps to obtain the data query results: Syntax parsing: The database management system first parses the query statement, checking its syntax for correctness and compliance with the database's SQL standards. Query plan generation: Based on the query statement structure and the preset database's metadata information, the database management system generates an optimal query execution plan to execute the query with minimal I / O operations and maximum speed. Data retrieval: According to the query plan, the database management system retrieves data rows from the storage system that meet the query conditions. Result sorting and aggregation: The retrieved data may require sorting or aggregation operations, such as sorting by date, grouping by item, and calculating averages. Result set return: Finally, the database management system returns the processed result set to the intelligent agent or the system component that initiated the query.

[0114] The data query results specifically contain the actual data referenced by the query configuration metrics in the user's request. For example, for the request above, the data query results could include the total number of code commits for project Y last month. The result set can be a single number, a data table, or a set of statistical values, depending on the requirements of the query configuration metrics. Furthermore, the result set can also include additional metadata, such as query time and the number of rows returned, which helps in evaluating the efficiency and performance of the query.

[0115] Step S204: Analyze the data query results based on the analysis configuration indicators to obtain the data analysis results.

[0116] Alternatively, in software development efficiency measurement scenarios, data analysis results are insights derived from in-depth analysis of data query results based on user-specified analysis configuration indicators. These insights can take various forms, such as statistical data, trend analysis, comparative analysis, predictive models, and defect root cause analysis, and are used to reveal efficiency, quality, and potential problems in the software development process.

[0117] For example, if the analysis of configuration metrics is about analyzing the code commit trends and defect rate changes of two development teams over the past three months, the data analysis results can include: trend graphs of code commit volume of team A and team B over time to show the activity of code commits; comparative analysis of defect rate changes over time to indicate which team's code quality is more stable; frequency and scale of code changes to calculate expected future defect rates to provide risk warnings; the main sources of code defects and propose targeted improvement suggestions.

[0118] The steps described above, through intelligent analytics, extract valuable information and insights from massive amounts of raw data. This helps software development teams promptly identify potential issues related to work efficiency and product quality, providing data support for process optimization and performance improvement. Furthermore, the development of personalized data analytics capabilities allows the implementation system to flexibly respond to the specific analytical needs of different users, enhancing the relevance and practicality of data analysis.

[0119] Optionally, in the data analysis method provided in this embodiment of the invention, the step of analyzing the data query results based on the analysis configuration indicators to obtain the data analysis results includes: determining the analysis target and analysis process based on the analysis configuration indicators, wherein the analysis process includes at least: data feature insight, data trend analysis, and data anomaly detection; constructing a data analysis workflow based on the analysis target and analysis process; inputting the data query results into the data analysis workflow and outputting an analysis report; and displaying the analysis report on the interactive interface to obtain the data analysis results.

[0120] It's important to note that the analysis objective refers to the specific insights or results that the target user hopes to gain from analyzing the data query results. This could be a deeper understanding of a particular characteristic of an indicator (such as the distribution of lines of code), an understanding of the trend of an indicator over time (such as trend analysis of code commit frequency), or the detection of abnormal data points (such as a sudden increase in defect density). The analysis objective helps the intelligent analysis agent to perform targeted analysis, ensuring that the results are highly relevant to the user's needs.

[0121] The data analysis process comprises at least three main stages: data feature insight, data trend analysis, and data anomaly detection. These three stages focus on detailed understanding of the data, time series analysis, and outlier identification, respectively, collectively forming a comprehensive data analysis perspective. Data feature insight focuses on understanding the basic attributes and internal structure of the data, such as the distribution of indicator values, the location of peak points, and differences between different categories. This stage can be based on statistical and probability theories, using methods such as histograms, box plots, and cluster analysis. Data trend analysis focuses on the patterns of data change over time, identifying trends such as growth, decline, or cyclical changes. Trend analysis can utilize time series analysis techniques, such as exponential smoothing or seasonality decomposition, to help reveal the long-term trend of the data. Data anomaly detection aims to identify data points or sequences that deviate from normal patterns, which is crucial for discovering potential problems or anomalies. Anomaly detection can be based on statistical threshold rules, machine learning algorithms, or deep learning models.

[0122] The steps for building a data analysis workflow can be as follows: First, based on the user-specified analysis objectives and preset analysis processes, the intelligent agent builds or invokes appropriate data analysis algorithms and tools; second, data preprocessing strategies are determined, such as handling missing values ​​and data standardization; next, a workflow diagram is constructed, connecting the aforementioned algorithms and tools in a logical order to form an analysis pipeline; finally, data query results are sent to different stages of the pipeline as needed, gradually generating analysis reports. In principle, the design of a data analysis workflow is based on a combination of data flow and algorithms, with the core being the orderly connection of a series of data processing and analysis steps to achieve automated analysis. The construction of the workflow relies on the configuration of the intelligent agent to ensure that each analysis node can produce its results based on the output of the previous node, while also considering the priority and complexity of the analysis objectives.

[0123] Data analysis workflows can take the form of algorithm call sequences, graphical flowcharts, or configuration files. In practice, a workflow can be represented as a configurable workflow engine, allowing target users or system administrators to define each step through an interface or scripts. During processing, data query results first enter a data preprocessing stage for standardization and cleaning; then, the feature insight engine analyzes the data's distribution and structure; next, the data is passed to a trend analysis engine to identify trends using time series analysis methods; finally, an anomaly detection engine runs to detect any unexpected behavior or patterns. The results of each stage are recorded as input for subsequent analysis or report generation.

[0124] Ultimately, the results of all analytical phases are integrated into a single analysis report. This process may also include using natural language generation techniques to transform the data analysis results into human-readable text descriptions. Report generation can be handled by a dedicated component, ensuring that the report is formatted consistently, rich in content, and easy to understand.

[0125] The analysis report includes the following sections: Overview: A brief overview of the analysis objectives and key findings. Detailed Analysis: Showcasing the results of data feature insights, data trend analysis, and data anomaly detection, potentially including charts, statistics, and text descriptions. Insight Summary: Key insights extracted from the analysis results, highlighting the crucial information behind the data. Recommendations and Action Plans: Optimization suggestions or next steps based on the data analysis results to help users take concrete actions.

[0126] Furthermore, presenting analytics reports not only provides visualized data analysis results, making it easier for users to intuitively understand the story behind the data, but also enhances users' trust in the intelligent analytics system, enabling target users to make more informed decisions based on data insights. In addition, presenting reports is also an important channel for the intelligent analytics system to communicate with users, promoting continuous system improvement and increasing user satisfaction. Besides analytics reports, intelligent analytics systems can also display: Real-time dashboards: providing dynamically updated viewpoints to help users monitor key performance indicators in real time; Interactive charts: allowing users to independently filter and explore data, discovering patterns or trends hidden within it; Data storyboards: telling the story behind the data through a series of charts and narrative text, enhancing the readability and appeal of the analysis.

[0127] Through the above steps S201 to S204, the target user's data query request and data analysis requirements input in natural language can be received through the interactive interface. Then, the data query request and data analysis requirements are parsed to obtain query configuration indicators and analysis configuration indicators. Then, the preset database is queried according to the query configuration indicators to obtain the data query results. The preset database is updated in the data background at a preset frequency. Finally, the data query results are analyzed according to the analysis configuration indicators to obtain the data analysis results.

[0128] In this embodiment of the invention, an intelligent agent combined with natural language processing is employed. By constructing a dynamic domain knowledge base and a deep learning model, the invention achieves the goal of automatically parsing and executing user natural language data query and analysis requests. This significantly improves the agility and efficiency of enterprise data analysis. Specifically, this embodiment utilizes a pre-connected domain knowledge base and deep learning technology in the intelligent agent to accurately understand and convert user data requests in everyday spoken language, generating precise query and analysis configuration indicators. This avoids the tedious manual configuration process of traditional methods. Simultaneously, the pre-set database is automatically updated periodically to ensure data timeliness and consistency. More importantly, this embodiment uses a deep learning model to semantically understand user query and analysis requests, achieving efficient conversion of natural language into database query statements. It can also adjust analysis strategies based on analysis configuration indicators and perform deep analysis feedback to the user, greatly simplifying the data analysis process and improving analysis accuracy. Through these steps, the workload of data analysts is significantly reduced, enabling enterprises to make more timely and accurate decisions. This solves the technical problem of cumbersome enterprise data analysis processes leading to low data analysis efficiency in related technologies.

[0129] The invention will now be described in conjunction with another alternative embodiment.

[0130] Example 2

[0131] This invention also provides a data analysis device. It should be noted that the data analysis device of this invention includes multiple implementation units, which can be used to execute the data analysis method provided in the first embodiment above. Each implementation unit corresponds to each implementation step in the first embodiment above.

[0132] Figure 3 This is a schematic diagram of an optional data analysis apparatus according to an embodiment of the present invention, such as... Figure 3 As shown, the device may include: a receiving unit 31, a parsing unit 32, a querying unit 33, and an analysis unit 34.

[0133] The receiving unit 31 is used to receive data query requests and data analysis needs input by the target user in natural language form through the interactive interface.

[0134] Parsing unit 32 is used to parse data query requests and data analysis requirements to obtain query configuration indicators and analysis configuration indicators;

[0135] The query unit 33 is used to query the preset database according to the query configuration indicators and obtain the data query results. The preset database is updated in the data background at a preset frequency.

[0136] Analysis unit 34 is used to analyze the data query results based on the analysis configuration indicators to obtain data analysis results.

[0137] The aforementioned data analysis device can first receive data query requests and data analysis requirements input by the target user in natural language form through the receiving unit 31 via an interactive interface. Then, the parsing unit 32 parses the data query requests and data analysis requirements to obtain query configuration indicators and analysis configuration indicators. Next, the query unit 33 queries a preset database based on the query configuration indicators to obtain data query results. The preset database is updated in the data background at a preset frequency. Finally, the analysis unit 34 analyzes the data query results based on the analysis configuration indicators to obtain data analysis results.

[0138] In this embodiment of the invention, an intelligent agent combined with natural language processing is employed. By constructing a dynamic domain knowledge base and a deep learning model, the goal of automating the parsing and execution of user natural language data query and analysis requests is achieved. This significantly improves the agility and efficiency of enterprise data analysis. Specifically, this embodiment utilizes a pre-connected domain knowledge base and deep learning technology in the intelligent agent to accurately understand and convert user data requests in everyday spoken language, generating precise query and analysis configuration indicators. This avoids the tedious manual configuration process of traditional methods. Simultaneously, the pre-set database is automatically updated periodically to ensure data timeliness and consistency. More importantly, this embodiment uses a deep learning model to semantically understand user query and analysis requests, achieving efficient conversion of natural language into database query statements. It can also adjust analysis strategies based on analysis configuration indicators and perform deep analysis feedback to the user, greatly simplifying the data analysis process and improving analysis accuracy. Through these steps, not only is the workload of data analysts significantly reduced, but enterprises can also make more timely and accurate decisions, thus solving the technical problem of cumbersome enterprise data analysis processes leading to low data analysis efficiency in related technologies.

[0139] Furthermore, the parsing unit includes: a retrieval module, used to retrieve the input interface of the target agent, wherein the target agent is pre-connected to the domain knowledge base and deep learning model, and the target agent is used to understand natural language and convert it into computer language based on the domain knowledge base and deep learning model; and a generation module, used to send data query requests and data analysis requests to the input interface, and generate query configuration indicators and analysis configuration indicators through the target agent.

[0140] Furthermore, the generation module includes: a semantic parsing submodule, used to perform semantic parsing on data query requests and data analysis needs through a deep learning model to obtain query intent keywords and analysis target keywords; a first retrieval submodule, used to retrieve domain knowledge bases based on query intent keywords to obtain query configuration indicators that match the query intent; and a second retrieval submodule, used to retrieve domain knowledge bases based on analysis target keywords to obtain analysis configuration indicators that match the analysis target.

[0141] Furthermore, the data analysis device also includes: an acquisition module, used to acquire the architecture information of the latest updated preset database in the data backend, wherein the architecture information includes: database information, table information, and field information; a first construction module, used to construct a metadata index of the preset database based on the architecture information, and determine Q indicator entries based on the metadata index, wherein Q is a positive integer; and an integration module, used to integrate all indicator entries to obtain a domain knowledge base.

[0142] Furthermore, the data analysis device also includes: a data acquisition module, used to acquire data from N data sources based on a preset frequency to obtain data to be updated, where N is a positive integer; a preprocessing module, used to preprocess the data to be updated, wherein the preprocessing includes the following operations: data cleaning and format conversion; and an update module, used to perform data consistency checks on the data to be updated from different data sources, and update the preset database using the data to be updated if the data is consistent.

[0143] Furthermore, the query unit includes: a sending module, used to construct a database query statement based on query configuration indicators and send the database query statement to a preset database; and an execution module, used to execute the database query statement in the preset database and receive the data query results returned by the preset database.

[0144] Furthermore, the analysis unit includes: a determination module, used to determine the analysis objectives and analysis process based on the analysis configuration indicators, wherein the analysis process includes at least: data feature insight, data trend analysis, and data anomaly detection; a second construction module, used to construct a data analysis workflow based on the analysis objectives and analysis process; an input module, used to input the data query results into the data analysis workflow and output an analysis report; and a feedback module, used to feed the analysis report back to the interactive interface for display, thereby obtaining the data analysis results.

[0145] It should be noted that the receiving unit 31, parsing unit 32, querying unit 33, and analysis unit 34 mentioned above correspond to steps S201 to S204 in Embodiment 1. The instances and application scenarios implemented by the above units and corresponding steps are the same, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above modules or units can be hardware or software components stored in memory (e.g., memory 104) and processed by one or more processors (e.g., processors 102a, 102b, ..., 102n). The above modules or units can also be part of a device and run in the computer terminal 10 provided in Embodiment 1.

[0146] The invention will now be described in conjunction with another alternative embodiment.

[0147] Example 3

[0148] The present invention can also provide an electronic device. Figure 4 This is a structural block diagram of an electronic device for performing a data analysis method according to an embodiment of the present invention, such as... Figure 4 As shown, the electronic device may include: one or more ( Figure 4 (Only one is shown) Processor 402, memory 404, memory controller, and peripheral interface, wherein the peripheral interface is connected to the radio frequency module, audio module and display.

[0149] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the data analysis method and apparatus in this application embodiment. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby realizing the aforementioned data analysis method. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0150] The processor can access information and applications stored in memory via a transmission device to perform the following steps: receiving data query requests and data analysis requirements input by the target user in natural language through an interactive interface; parsing the data query requests and data analysis requirements to obtain query configuration indicators and analysis configuration indicators; querying a preset database based on the query configuration indicators to obtain data query results, wherein the preset database is updated in the data background at a preset frequency; and analyzing the data query results based on the analysis configuration indicators to obtain data analysis results.

[0151] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: calling the input interface of the target agent, wherein the target agent is pre-connected to the domain knowledge base and deep learning model, and the target agent is used to understand natural language and convert it into computer language based on the domain knowledge base and deep learning model; sending data query requests and data analysis requests to the input interface, and generating query configuration indicators and analysis configuration indicators through the target agent.

[0152] The processor can also access information and applications stored in the memory via a transmission device to perform the following steps: semantically parsing data query requests and data analysis needs using a deep learning model to obtain query intent keywords and analysis target keywords; retrieving domain knowledge bases based on query intent keywords to obtain query configuration indicators that match the query intent; and retrieving domain knowledge bases based on analysis target keywords to obtain analysis configuration indicators that match the analysis target.

[0153] The processor can also access information and applications stored in the memory via a transmission device to perform the following steps: obtain the latest updated architecture information of the preset database from the data backend, wherein the architecture information includes: database information, table information, and field information; construct a metadata index of the preset database based on the architecture information, and determine Q indicator entries based on the metadata index, wherein Q is a positive integer; integrate all indicator entries to obtain a domain knowledge base.

[0154] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: collect data from N data sources based on a preset frequency to obtain the data to be updated, where N is a positive integer; preprocess the data to be updated, where the preprocessing includes the following operations: data cleaning and format conversion; perform data consistency checks on the data to be updated from different data sources, and update the preset database using the data to be updated if the data is consistent.

[0155] The processor can also invoke information and applications stored in the memory via a transmission device to perform the following steps: constructing a database query statement based on query configuration indicators and sending the database query statement to a preset database; executing the database query statement in the preset database and receiving the data query results returned by the preset database.

[0156] The processor can also access information and applications stored in the memory via a transmission device to perform the following steps: determine the analysis objectives and analysis process based on the analysis configuration indicators, wherein the analysis process includes at least: data feature insight, data trend analysis, and data anomaly detection; construct a data analysis workflow based on the analysis objectives and analysis process; input the data query results into the data analysis workflow and output an analysis report; and display the analysis report on the interactive interface to obtain the data analysis results.

[0157] This invention provides a data analysis solution. By combining intelligent agents with natural language processing, and through the construction of a dynamic domain knowledge base and deep learning models, it achieves the goal of automatically parsing and executing user natural language data query and analysis requests. This significantly improves the agility and efficiency of enterprise data analysis. Specifically, this invention utilizes a pre-connected domain knowledge base and deep learning technology in the intelligent agent to accurately understand and convert user data requests in everyday spoken language, generating precise query and analysis configuration indicators. This avoids the tedious manual configuration process of traditional methods. Simultaneously, the pre-set database is automatically updated periodically to ensure data timeliness and consistency. More importantly, this invention uses deep learning models to semantically understand user query and analysis requests, achieving efficient conversion of natural language into database query statements. It can also adjust analysis strategies based on analysis configuration indicators and perform deep analysis feedback to the user, greatly simplifying the data analysis process and improving analysis accuracy. Through these steps, it not only significantly reduces the workload of data analysts but also enables enterprises to make more timely and accurate decisions, thus solving the technical problem of cumbersome enterprise data analysis processes leading to low data analysis efficiency in related technologies.

[0158] Those skilled in the art will understand that Figure 4 The structure shown is for illustrative purposes only. Electronic devices can also be smartphones, tablets, handheld computers, mobile internet devices (MIDs), PADs, and other terminal devices. Figure 4 This does not limit the structure of the aforementioned electronic device. For example, electronic devices may also include components that are more... Figure 4 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 4 The different configurations shown.

[0159] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0160] The invention will now be described in conjunction with another alternative embodiment.

[0161] Example 4

[0162] This invention also provides a computer-readable storage medium. Optionally, in this invention, the computer-readable storage medium can be used to store the program code executed by the data analysis method provided in Embodiment 1.

[0163] Optionally, in this embodiment of the invention, the storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0164] This invention also provides a computer program product, which, when executed on a data processing device, is suitable for performing data analysis method steps: receiving data query requests and data analysis requirements input by a target user in natural language form through an interactive interface; parsing the data query requests and data analysis requirements to obtain query configuration indicators and analysis configuration indicators; querying a preset database according to the query configuration indicators to obtain data query results, wherein the preset database is updated in the data background at a preset frequency; and analyzing the data query results according to the analysis configuration indicators to obtain data analysis results.

[0165] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0166] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0167] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0168] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0169] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0170] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0171] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A data analysis method, characterized in that, include: The system receives data query requests and data analysis needs from target users in natural language through an interactive interface. The data query request and the data analysis requirements are parsed to obtain query configuration indicators and analysis configuration indicators; The preset database is queried according to the query configuration indicators to obtain the data query results. The preset database is updated in the data background at a preset frequency. The data query results are analyzed based on the aforementioned analysis configuration indicators to obtain data analysis results.

2. The data analysis method according to claim 1, characterized in that, The steps of parsing the data query request and the data analysis requirements to obtain query configuration indicators and analysis configuration indicators include: The input interface of the target agent is invoked, wherein the target agent is pre-connected to a domain knowledge base and a deep learning model, and the target agent is used to understand natural language and convert it into computer language based on the domain knowledge base and the deep learning model; The data query request and the data analysis request are sent to the input interface, and the query configuration indicators and the analysis configuration indicators are generated through the target proxy.

3. The data analysis method according to claim 2, characterized in that, The steps of generating the query configuration metrics and the analysis configuration metrics through the target proxy include: The deep learning model is used to perform semantic parsing on the data query request and the data analysis requirements to obtain query intent keywords and analysis target keywords. Based on the query intent keywords, the domain knowledge base is retrieved to obtain the query configuration indicators that match the query intent; Based on the keywords of the analysis target, the domain knowledge base is searched to obtain the analysis configuration indicators that match the analysis target.

4. The data analysis method according to claim 2, characterized in that, The domain knowledge base is constructed through the following steps: Obtain the latest updated architecture information of the preset database from the data backend, wherein the architecture information includes: database information, table information, and field information; Based on the architecture information, construct the metadata index of the preset database, and determine Q indicator entries based on the metadata index, where Q is a positive integer; The domain knowledge base is obtained by integrating all the aforementioned indicator entries.

5. The data analysis method according to claim 1, characterized in that, The update steps of the preset database include: Data is collected from N data sources based on the preset frequency to obtain the data to be updated, where N is a positive integer; The data to be updated is preprocessed, wherein the preprocessing includes the following operations: data cleaning and format conversion; A data consistency check is performed on the data to be updated from different data sources. If the data is consistent, the preset database is updated using the data to be updated.

6. The data analysis method according to claim 1, characterized in that, The steps for querying a preset database based on the specified query configuration indicators to obtain data query results include: A database query statement is constructed based on the query configuration indicators, and the database query statement is sent to the preset database; Execute the database query statement in the preset database and receive the data query results returned by the preset database.

7. The data analysis method according to claim 1, characterized in that, The steps for analyzing the data query results based on the aforementioned analysis configuration indicators to obtain data analysis results include: The analysis objectives and analysis process are determined based on the analysis configuration indicators, wherein the analysis process includes at least: data feature insight, data trend analysis, and data anomaly detection; Construct a data analysis workflow based on the analysis objectives and the analysis process; The data query results are input into the data analysis workflow, and an analysis report is output. The analysis report is then displayed on the interactive interface to obtain the data analysis results.

8. A data analysis device, characterized in that, include: The receiving unit is used to receive data query requests and data analysis needs input by the target user in natural language form through an interactive interface. The parsing unit is used to parse the data query request and the data analysis requirements to obtain query configuration indicators and analysis configuration indicators; The query unit is used to query a preset database according to the query configuration indicators to obtain data query results, wherein the preset database is updated in the data background at a preset frequency; The analysis unit is used to analyze the data query results based on the analysis configuration indicators to obtain data analysis results.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the data analysis method according to any one of claims 1 to 7.

10. An electronic device, characterized in that, It includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the data analysis method according to any one of claims 1 to 7.

11. A computer program product, characterized in that, It includes computer instructions, wherein when executed by a processor, the computer instructions implement the steps of the data analysis method according to any one of claims 1 to 7.

Images