Dify platform-based intelligent big data summarization system for dialogue interaction

Abstract

The invention relates to the technical field of big data analysis, and discloses a dialogue type interaction intelligent big data summarization system based on a Dify platform, which comprises a front-end interaction module, a rear-end service module and a workflow arrangement module. According to the system, conversational interaction between a user and the system is realized through a natural language processing technology, user requirements are automatically analyzed, and an accurate data analysis task is generated. The system dynamically selects the most suitable model for data processing and summarization by integrating the generative AI model, so that the analysis efficiency and precision are improved. Compared with a traditional big data analysis tool, the method has the advantages that the development cost and the technical threshold are remarkably reduced, and a user can quickly obtain a high-quality analysis result without mastering complex data analysis knowledge. The system can flexibly respond according to natural language input of a user, automatically execute data query, processing and summarization tasks, realize automation and intelligence of a data analysis process, and greatly improve popularization and usability of data analysis.

Description

Technical Field

[0001] The present invention relates to a conversational interactive intelligent big data summarization system based on a Dify platform, belonging to the technical field of big data analysis. Background Art

[0002] With the rapid development of big data, data analysis has become an important basis for corporate decision-making. Traditional big data analysis tools usually rely on static pages and chart displays, and users need to obtain data insights through charts, reports, etc. These tools usually require users to have a certain technical background or data analysis knowledge, and the operation is complicated and not intuitive, resulting in a high threshold for users within the enterprise to use these tools, which restricts the popularization of data analysis.

[0003] In the existing technology, although most tools provide visual analysis functions, their limitations are still significant: first, the development cost is high. Enterprises need to invest a lot of manpower and time in front-end development to meet different data requirements and display forms; second, the user experience is poor during use. Users need to frequently switch pages or chart tools to view data during operation, and the analysis process is cumbersome and inefficient; third, the system has poor flexibility. Every change in requirements requires redevelopment, resulting in insufficient adaptability and scalability of the system.

[0004] In order to avoid these problems, the industry has tried to improve the intelligence level of data analysis by integrating artificial intelligence technology, especially by simplifying user operations through conversational interaction (such as natural language processing), so that non-technical personnel can also participate in data analysis. However, this solution still faces the following problems: 1. Existing conversational interaction systems often lack sufficient intelligent support and cannot accurately generate data analysis tasks according to user intentions, resulting in users being unable to efficiently obtain the required results; 2. Traditional generative AI technology has not been effectively integrated into the workflow, resulting in the processing of complex data still relying on manual configuration and failing to fully realize automation; 3. The system's ability to respond to personalized needs is poor, and it is often unable to flexibly adjust the analysis process according to different business needs, resulting in insufficient flexibility.

[0005] Therefore, how to reduce development costs, improve user experience and realize intelligent data analysis has become a technical problem to be solved by the present invention. Summary of the invention

[0006] The present invention provides a conversational interactive intelligent big data summarization system based on the Dify platform, the main purpose of which is to solve the problems of high development cost, poor user experience and insufficient system flexibility.

[0007] To achieve the above-mentioned purpose, the present invention provides a conversational interactive intelligent big data summary system based on the Dify platform, including a front-end interaction module, a back-end service module, and a workflow orchestration and task execution module; The front-end interaction module is configured to receive a natural language query or analysis request input by a user, and transmit the request to the back-end service module in real time; The backend service module is configured to receive the natural language query or analysis request, identify the type of analysis requirement implied by the user based on semantic analysis of the request, and extract key parameters related to the type of analysis requirement; The backend service module is further configured to generate Dify platform executable instructions including task type, data range, and model selection parameters according to the identified analysis requirement type and extracted key parameters; The workflow arrangement and task execution module is a Dify platform, which is configured to receive the Dify platform executable instructions from the backend service module and dynamically generate a data analysis workflow according to the instructions, wherein the data analysis workflow includes a data query step, a data processing step, and a data summarization step; Among them, the data summarizing step is configured to dynamically select and configure a specific type of generative artificial intelligence model that matches the analysis requirement type and model selection parameters from a variety of different types of generative artificial intelligence models integrated on the Dify platform according to the analysis requirement type and model selection parameters contained in the instruction, use the selected specific type of generative artificial intelligence model to process and summarize the queried data, return the summary results to the back-end service module, and finally present them to the user through the front-end interaction module.

[0008] Preferably, the backend service module performs intent recognition on the natural language query or analysis request, including identifying the analysis requirement type of the user's request, such as message summary, data query, user experience element analysis, trend analysis or sentiment analysis.

[0009] Preferably, the generative artificial intelligence models integrated in the Dify platform include but are not limited to: a text summary model for message summarization, a time series analysis model for trend analysis, a sentiment classification model for sentiment analysis, and a specific domain knowledge model for user experience element analysis.

[0010] Preferably, after receiving the natural language query or analysis request, the backend service module is also configured to query the original data associated with the request from at least one preset data source, and encapsulate the queried original data into a unified structure and upload it to the cloud storage platform for the Dify platform to call when executing the data analysis workflow.

[0011] Preferably, the cloud storage platform is an object storage service (OSS) platform.

[0012] Preferably, the front-end interaction module is constructed using Web technology and supports real-time verification of user input and context retention functions.

[0013] Preferably, the backend service module and the frontend interaction module perform data transmission via a RESTful API, and after data summarization is completed, the summary results are returned to the frontend interaction module in real time via the server-sent event (SSE) technology for rendering and display.

[0014] Preferably, the visual orchestration tool provided by the Dify platform is used for developers to configure each step in the data analysis workflow, including data source connection, data processing logic, and selection and parameter configuration of generative artificial intelligence models.

[0015] Preferably, the intelligent big data summary system is applied to scenarios such as enterprise-level customer data analysis and report generation, big data monitoring and distribution trend prediction, customer experience management system or e-commerce customer service data analysis.

[0016] Preferably, the backend service module implements the intent recognition function based on a large language model of the Transformer architecture.

[0017] Compared with the problems described in the background technology, the beneficial effects of the present invention are: 1. By not only identifying the user's general analysis needs, but more importantly extracting the specific parameters contained in their natural language input (such as time range, specific entities, and analysis dimensions), the system can dynamically select and configure the generative AI model that best fits the parameters within the Dify platform. This parameter-driven refined selection mechanism ensures that the data summary can be highly consistent with the user's specific analysis intent, thereby generating more accurate and practical insights than the results obtained by general or untargeted AI models. For example, when analyzing user feedback, for negative reviews about product X in the past week, the system can call a model that is optimized for sentiment analysis for a specific time period and product, thereby more accurately identifying key pain points.

[0018] 2. The system cleverly combines the dynamic workflow generation capabilities of the Dify platform with the selection of specific AI models based on user intent and parameters, achieving a high degree of automation and intelligence in the analysis process. The dynamically generated workflow can flexibly arrange data query, processing, and summarization steps according to user needs, while the customized AI model selection ensures that each step is performed by the most appropriate tool. This synergy significantly improves the overall analysis efficiency. Users can quickly obtain highly relevant analysis results without tedious manual configuration, greatly shortening the time from data to insight.

[0019] 3. The system can understand the analysis requirements and key parameters implied in the user's natural language input, so that users can perform data analysis just like communicating with people without having to learn complex query syntax or operation processes. The system's deep understanding of the context enables it to dynamically adjust the analysis strategy and select the most appropriate AI model based on the user's specific questions, thereby providing a smarter and more personalized analysis experience, lowering the user threshold, and improving the popularity and ease of use of data analysis.

[0020] 4. By dynamically selecting and configuring AI models based on the uniqueness of each user request, the system demonstrates excellent flexibility and adaptability. Whether it is trend prediction, sentiment analysis, user experience factor analysis, or other specific types of analysis needs, the system is able to select the most appropriate AI model for processing based on the specific situation. This flexibility enables the system to respond to changing analysis needs and diverse application scenarios, providing continuous value to users.

[0021] 5. The system does not rely on a single general AI model to handle all analytical tasks, but can intelligently call various specialized AI models integrated in the Dify platform. This on-demand calling method ensures that the most appropriate tool can be used for a specific task, thereby avoiding resource waste and improving computing efficiency, especially when processing large-scale, multi-dimensional data, and can more efficiently complete data summarization and analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a schematic diagram of the front-end interaction page of the present invention.

[0023] Figure 2 It is the dialogue interface of the intelligent CEM (customer experience management) system of the present invention.

[0024] Figure 3 This is a flow chart of the conversational interactive intelligent big data summary system based on the Dify platform of the present invention.

[0025] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings in conjunction with the embodiments. DETAILED DESCRIPTION

[0026] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0027] The embodiment of the present application provides a conversational interactive intelligent big data summary system based on the Dify platform, including a front-end interaction module, a back-end service module, and a workflow orchestration and task execution module; The front-end interaction module is configured to receive a natural language query or analysis request input by a user, and transmit the request to the back-end service module in real time; The backend service module is configured to receive the natural language query or analysis request, identify the type of analysis requirement implied by the user based on semantic analysis of the request, and extract key parameters related to the type of analysis requirement; The backend service module is further configured to generate Dify platform executable instructions including task type, data range, and model selection parameters according to the identified analysis requirement type and extracted key parameters; The workflow arrangement and task execution module is a Dify platform, which is configured to receive the Dify platform executable instructions from the backend service module and dynamically generate a data analysis workflow according to the instructions, wherein the data analysis workflow includes a data query step, a data processing step, and a data summarization step; Among them, the data summarizing step is configured to dynamically select and configure a specific type of generative artificial intelligence model that matches the analysis requirement type and model selection parameters from a variety of different types of generative artificial intelligence models integrated on the Dify platform according to the analysis requirement type and model selection parameters contained in the instruction, use the selected specific type of generative artificial intelligence model to process and summarize the queried data, return the summary results to the back-end service module, and finally present them to the user through the front-end interaction module.

[0028] Preferably, the backend service module performs intent recognition on the natural language query or analysis request, including identifying the analysis requirement type of the user's request, such as message summary, data query, user experience element analysis, trend analysis or sentiment analysis.

[0029] Preferably, the generative artificial intelligence models integrated in the Dify platform include but are not limited to: a text summary model for message summarization, a time series analysis model for trend analysis, a sentiment classification model for sentiment analysis, and a specific domain knowledge model for user experience element analysis.

[0030] Preferably, after receiving the natural language query or analysis request, the backend service module is also configured to query the original data associated with the request from at least one preset data source, and encapsulate the queried original data into a unified structure and upload it to the cloud storage platform for the Dify platform to call when executing the data analysis workflow.

[0031] Preferably, the cloud storage platform is an object storage service (OSS) platform.

[0032] Preferably, the front-end interaction module is constructed using Web technology and supports real-time verification of user input and context retention functions.

[0033] Preferably, the backend service module and the frontend interaction module perform data transmission via a RESTful API, and after data summarization is completed, the summary results are returned to the frontend interaction module in real time via the server-sent event (SSE) technology for rendering and display.

[0034] Preferably, the visual orchestration tool provided by the Dify platform is used for developers to configure each step in the data analysis workflow, including data source connection, data processing logic, and selection and parameter configuration of generative artificial intelligence models.

[0035] Preferably, the intelligent big data summary system is applied to scenarios such as enterprise-level customer data analysis and report generation, big data monitoring and distribution trend prediction, customer experience management system or e-commerce customer service data analysis.

[0036] Preferably, the backend service module implements the intent recognition function based on a large language model of the Transformer architecture.

[0037] See also Figure 1The figure shows the conversation interface of the intelligent CEM (customer experience management) system. Users interact with the AI ​​assistant through this interface. At the top of the interface, the icon and name of the Cloud Listening Assistant AI are displayed, indicating that this is the intelligent assistant function of the system. In the dialog box, the system first introduces the function of the intelligent CEM to the user: "Start your journey to intelligent CEM", and provides a function prompt, pointing out that the system can recognize the user's demand messages and can include customer comments, customer service conversations, work orders, social media data and other content for summary and analysis. The button "Go to use" below allows users to directly enter the system to perform relevant operations. Clicking the "Learn how AI can help you" link, users can get more instructions on how AI can help process and analyze CEM data. The input box at the bottom encourages users to directly enter words to talk to AI, prompting users to let AI help solve related problems through dialogue. Through this interactive design, the system improves the user experience, simplifies the complexity of data query and analysis, and enables non-technical personnel to easily use the system to analyze and summarize customer experience data.

[0038] See also Figure 2 ,The figure shows the intelligent CEM (customer experience management) system dialogue interface, specifically the analysis process when the user interacts with the AI ​​assistant. In this interface, the AI ​​assistant first shows the user that the current dialogue has not been filtered, and the page data shows a total of 89,695 items, including customer service conversation data and other data types, and prompts the user to select the data type through the dialogue for further analysis.

[0039] The AI ​​assistant suggests that if the user has determined the specific content of the analysis, they can directly tell the AI, or choose to directly specify the data range for summary, and the system will perform precise analysis based on these instructions. In addition, users can try to enter requests like "Please summarize using the Kano analysis framework" or "Please analyze the user experience elements (5E) analysis" to start a more specific analysis.

[0040] In the following conversation, the AI ​​assistant explained in detail how to use the 5E analysis framework for user experience. AI mentioned that the 5E framework consists of five dimensions: Effective, Efficient, Easy to use, Enjoyable, and Emotional. Then, AI explained the meaning of each dimension, for example, effectiveness refers to the system's ability to answer customer questions, especially how to optimize product features or improve customer service.

[0041] See also Figure 3The workflow of the conversational interactive intelligent big data summary system based on the Dify platform is demonstrated. First, the front-end interaction module is responsible for receiving the user's natural language query request and passing it to the back-end service module. Then, the back-end service module performs semantic analysis on the natural language query, identifies the analysis requirement type and parameters in the query, and generates the corresponding Dify platform executable instructions. Then, the system dynamically generates a data analysis workflow based on these instructions by the workflow orchestration and task execution module, which includes three main steps: data query, data processing, and data summary. In the data query stage, the system obtains raw data from relevant data sources; in the data processing stage, the system processes the queried data; in the data summary stage, the system summarizes the data according to the analysis requirements and the selected model type through the generative AI model, generates analysis results, and returns them to the back-end service module. Finally, the results are displayed to the user through the front-end interaction module. The design of this process makes the data analysis process automated and intelligent, greatly reducing the complexity of user operations and improving analysis efficiency.

[0042] Example 1: In the customer experience management (CEM) system of a large retail enterprise, data analysis is the key to optimizing services, improving customer satisfaction and promoting sales. Traditional customer feedback analysis methods mainly rely on static reports and charts, and account managers need to manually analyze customer feedback, organize data and summarize valuable information. However, due to the problems of cumbersome user operations, single data display form, and low analysis efficiency of traditional methods, account managers are often unable to quickly extract key trends and insights from a large amount of customer feedback, resulting in delayed decision-making and low efficiency.

[0043] The company introduced a conversational interactive intelligent big data summary system based on the Dify platform. Through the integration of the system, account managers only need to interact with the system through natural language to automatically obtain real-time analysis results of customer feedback data. For example, the account manager enters a sentence through the front-end interactive module: "Summarize all customer feedback about product A in the past month." The system analyzes the account manager's needs through natural language processing (NLP) technology, quickly identifies key information such as the past month, product A, and customer feedback, and converts it into specific analysis instructions.

[0044] At this point, the system uses the workflow orchestration function of the Dify platform to break down the instruction into specific task steps, including data query, processing, and summary. Based on the preset task template, the system automatically extracts all relevant comments about Product A in the past month from the customer feedback data stored in the enterprise cloud, performs sentiment analysis, keyword extraction, trend analysis, and other processing, and uses the generative AI model to make a concise message summary of the processed results. The summary results include: the overall customer satisfaction of Product A has declined, the main problem is concentrated in the "quality" aspect, and the main source of negative comments is the "price" factor.

[0045] In this process, the system's intelligent analysis has greatly improved work efficiency. Account managers no longer need to manually organize and analyze data, nor rely on cumbersome chart displays. Through natural language dialogue with the system, account managers can obtain accurate and personalized analysis results in real time, directly guiding subsequent decisions and actions. In addition, because the system can automatically select the most appropriate AI model based on the customer's specific questions and flexibly respond to different analysis needs, enterprises can achieve rapid transformation from data to insights, greatly improving analysis efficiency and the timeliness of decision-making.

[0046] Compared with traditional manual analysis methods, this invention reduces manual intervention and the possibility of analysis errors through automated task scheduling and intelligent data summarization. Secondly, the generative AI model can extract key information from massive data and generate more accurate and efficient reports. Finally, through deep integration with the Dify platform, enterprises can quickly deploy and apply this intelligent analysis tool without investing a lot of development costs, thereby realizing the automation and intelligence of data analysis.

[0047] Example 2: In the market research department of a medium-sized e-commerce company, the system needs to analyze multi-dimensional market data in real time to adjust product planning and market strategies in a timely manner. Traditional market research methods mainly rely on manual data collection and analysis, and usually present results through Excel spreadsheets and manually generated reports. This process is not only time-consuming and laborious, but also due to the huge amount of data and complex analysis dimensions, manual analysis often cannot accurately grasp market dynamics, resulting in delayed decision-making.

[0048] The company uses a conversational interactive intelligent big data summary system based on the Dify platform. Through this system, market researchers can interact with the system using natural language, which greatly simplifies the data analysis process. For example, market researchers input through the front-end interactive module: "Please analyze the sales trends of major e-commerce platforms in the past three months and give keywords." The system quickly identifies key information such as the past three months, e-commerce platforms, sales trends and keywords through natural language processing technology, and converts them into data query and analysis instructions.

[0049] In the backend service module of the system, the Dify platform automatically configures data query tasks based on the parsed user needs, connects to the data sources of multiple e-commerce platforms, and retrieves the corresponding sales data. Subsequently, the system conducts in-depth analysis of the query results through the integrated generative AI model, such as time series trend analysis, keyword extraction, etc., and finally generates a concise report. The report lists the sales trends of major e-commerce platforms and extracts keywords related to sales changes, such as promotional activities, holidays, and new product launches.

[0050] In this way, market researchers can not only quickly obtain multi-dimensional analysis results, but also flexibly adjust the dimensions and focus of the analysis. Traditional report generation may take several days, but the system can complete the same task in a few minutes, significantly improving the efficiency of market research. More importantly, the system can dynamically adjust the analysis method according to different analysis needs, making the results more accurate and personalized, helping companies to respond quickly to market changes. Through the combination of dialogue-based interaction and generative AI technology, companies can obtain market trend reports in real time and accurately without relying on traditional complex analysis tools, thereby improving the timeliness and accuracy of decision-making. In addition, the system can automatically select the most appropriate analysis model based on user input, avoiding the complex process of manually selecting models and setting parameters, saving a lot of time and cost for enterprises, and significantly improving the level of intelligence of data analysis, all of which belong to extended implementation methods known to ordinary technicians in this field.

[0051] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0052] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A conversational interactive intelligent big data summary system based on the Dify platform, including a front-end interaction module, a back-end service module, and a workflow orchestration and task execution module, characterized by: The front-end interaction module is configured to receive a natural language query or analysis request input by a user, and transmit the request to the back-end service module in real time; The backend service module is configured to receive the natural language query or analysis request, identify the type of analysis requirement implied by the user based on semantic analysis of the request, and extract key parameters related to the type of analysis requirement; The backend service module is further configured to generate Dify platform executable instructions including task type, data range, and model selection parameters according to the identified analysis requirement type and extracted key parameters; The workflow arrangement and task execution module is a Dify platform, which is configured to receive the Dify platform executable instructions from the backend service module and dynamically generate a data analysis workflow according to the instructions, wherein the data analysis workflow includes a data query step, a data processing step, and a data summarization step; Among them, the data summarizing step is configured to dynamically select and configure a specific type of generative artificial intelligence model that matches the analysis requirement type and model selection parameters from a variety of different types of generative artificial intelligence models integrated on the Dify platform according to the analysis requirement type and model selection parameters contained in the instruction, use the selected specific type of generative artificial intelligence model to process and summarize the queried data, return the summary results to the back-end service module, and finally present them to the user through the front-end interaction module.

2. The intelligent big data summarization system according to claim 1, characterized in that: The backend service module performs intent recognition on the natural language query or analysis request, including identifying the analysis requirement type of the user's request, such as message summary, data query, user experience factor analysis, trend analysis or sentiment analysis.

3. The intelligent big data summarization system according to claim 2, characterized in that: The generative artificial intelligence models integrated in the Dify platform include, but are not limited to: a text summary model for message summarization, a time series analysis model for trend analysis, a sentiment classification model for sentiment analysis, and a specific domain knowledge model for user experience element analysis.

4. The intelligent big data summarization system according to claim 1, characterized in that: After receiving the natural language query or analysis request, the backend service module is also configured to query the original data associated with the request from at least one preset data source, and encapsulate the queried original data into a unified structure and upload it to the cloud storage platform for the Dify platform to call when executing the data analysis workflow.

5. The intelligent big data summarization system according to claim 4 is characterized in that: The cloud storage platform is an object storage service OSS platform.

6. The intelligent big data summarization system according to claim 1, characterized in that: The front-end interaction module is constructed using Web technology and supports real-time verification of user input and context retention functions.

7. The intelligent big data summarization system according to claim 1, characterized in that: The backend service module and the frontend interaction module perform data transmission via a RESTful API, and after data summary is completed, the summary result is returned to the frontend interaction module in real time for rendering and display via the server-sent event SSE technology.

8. The intelligent big data summarization system according to claim 1, characterized in that: The visual orchestration tool provided by the Dify platform is used for developers to configure each step in the data analysis workflow, including data source connection, data processing logic, and selection and parameter configuration of generative artificial intelligence models.

9. The intelligent big data summarization system according to claim 1, characterized in that: The intelligent big data summary system is applied to scenarios of enterprise-level customer data analysis and report generation, big data monitoring and distribution trend prediction, customer experience management system or e-commerce customer service data analysis.

10. The intelligent big data summarization system according to claim 2, characterized in that: The backend service module implements the intent recognition function based on a large language model of the Transformer architecture.

Images