Customized plug-in driven data analysis method and system and storage medium

Abstract

The invention relates to a customized plug-in driven data analysis method and system and a storage medium, and aims to provide a flexible, efficient and extensible data analysis solution. According to the method, a specific data set is processed through a user-defined plug-in, and rapid analysis and deep mining of data are achieved. The system comprises a data input module, a plug-in management module, a data analysis module and a result output module, and supports various data formats and analysis requirements. The storage medium is used for storing related programs of the system.

Description

Technical Field

[0001] The present invention relates to the fields of data analysis technology and artificial intelligence, and in particular to a data analysis method and system capable of enhancing data processing capabilities through customized plug-ins, and a storage medium for storing related programs. Background Art

[0002] With the continuous development of data science, data analysis plays an increasingly important role in business decision-making, scientific research and daily life. Enterprises and organizations need to extract valuable information from massive data to support decision-making and business development. However, existing data analysis tools often have functional limitations and cannot meet users' customized needs for different business scenarios. In addition, expanding the functions of existing tools often requires professional programming skills, which makes it difficult for non-technical users to customize specific data analysis methods. To solve this problem, the present invention proposes a data analysis method, system and storage medium based on customized plug-in drive, aiming to provide a flexible and scalable data analysis solution to meet the specific data analysis needs of different users or scenarios.

[0003] LLM (Large Language Model) usually refers to large language models, which are trained using deep learning techniques, especially the Transformer architecture. They can process, understand and generate natural language texts and achieve excellent performance in a variety of language tasks, such as text generation, machine translation, question answering, text summarization, sentiment analysis, etc. Summary of the invention

[0004] The present invention provides a customized plug-in driven data analysis method, system and storage medium, which allow users to perform personalized data analysis tasks by configuring specific plug-ins. These plug-ins follow a unified interface standard and can run independently of the main program, thereby realizing rapid expansion and customization of data analysis functions. The flexibility of data analysis is enhanced and the complexity of user use is simplified.

[0005] The technical solution of the present invention is: to provide a plug-in definition method, comprising:

[0006] 1.1 Plug-in requirements analysis: Identify and clarify the purpose and function of the plug-in, determine the specific problems that the plug-in should solve, the expected behavior and performance requirements;

[0007] 1.2 Plug-in design: Create a plug-in architecture and design specification list, which clearly defines the plug-in's internal structure, interface definition, data flow, and how it interacts with the main program or other plug-ins;

[0008] 1.3 Plug-in interface specification formulation: define the standardized interface for communication between plug-ins and main programs or other plug-ins, including input and output parameters, calling protocols, and data formats;

[0009] 1.4 Programming implementation: Carry out plug-in coding according to design documents and interface specifications. Use appropriate programming languages ​​and tools to develop executable plug-in components;

[0010] 1.5 Test verification: Implement unit testing, functional testing, and performance testing to ensure that the plug-in works as expected and interacts and executes functions correctly in the main program;

[0011] 1.6 Plug-in release: Publish the compiled plug-in list to the plug-in library, provide detailed user manuals, installation guides and API documentation, so that developers and users can understand how to correctly install, configure and use the plug-in, and users can rate and give feedback on the plug-in;

[0012] 1.7 Maintenance and Updates: After the plugin is released, regular technical support is provided to maintain the plugin version, fix bugs found in the program, respond to user feedback, and improve performance and functionality. This plugin definition process helps to systematize and standardize the plugin development process, ensure the quality and compatibility of the results, and facilitate the long-term maintenance and iteration of the plugin.

[0013] Furthermore, the design specification list in the plug-in design step of step 1.2 includes a detailed description of the scalability, security and user customization options of the plug-in.

[0014] Furthermore, in the step 1.3 of formulating the plug-in interface specification, the interface specification is defined using the OpenAPI specification.

[0015] Furthermore, in the programming implementation step of step 1.4, the programming language used includes but is not limited to Java, Python, C++ or C#.

[0016] Furthermore, in the test verification step 1.5, test cases are automatically generated and cover all defined interface specifications.

[0017] Furthermore, in the step 1.6 of plug-in publishing, the plug-in library supports version control and historical version rollback functions.

[0018] Furthermore, in the maintenance and update step 1.7, the technical support includes remote diagnosis, performance tuning and provision of security patches.

[0019] The present invention also provides a computer system for executing the plug-in definition method, the computer system comprising a storage medium, at least one processor and a memory communicatively connected to the processor, the storage medium storing a computer program, and the computer program implementing the steps of the plug-in definition method when executed by the processor.

[0020] The present invention also provides a computer-readable storage medium storing a computer program, wherein the computer program implements the steps of the plug-in definition method when executed by a processor.

[0021] The present invention also provides a customized plug-in driven data analysis method, comprising the following steps:

[0022] 2.1 Build a basic data analysis platform, which has basic functional modules such as data access, preprocessing, storage, calculation and display;

[0023] 2.2 Design and develop a standardized plug-in platform to allow third-party developers to write customized plug-ins based on specific data analysis needs. The plug-ins contain core logic such as data processing algorithms, models, rules, etc., and are seamlessly connected to the data analysis basic platform through interfaces;

[0024] 2.3 Users search for plug-ins in the plug-in library based on functions, ratings, user feedback and other information according to actual needs, or customize required plug-ins; the plug-ins are pre-installed in the plug-in library, their functions are designed for specific data analysis tasks, and interact with other components of the data analysis system through well-defined interface protocols;

[0025] 2.4 Integrate the configured plug-in into the data analysis system so that the plug-in can access and process the user-specified data set;

[0026] 2.5 The user submits the requirements to be analyzed and selects the plug-ins required to meet the actual analysis requirements. The plug-ins can be one or more. While submitting the analysis requirements, the conversation context is maintained to maintain consistency and accuracy in multiple rounds of interactions.

[0027] 2.6 The system uses natural language processing technology to parse user needs, semantically matches the user's intentions and extracted entities with the custom plug-ins in the plug-in library, and finds the plug-in that is most suitable for executing the user's request.

[0028] 2.7 The data analysis control center calls the plug-in to query data from the data platform, perform data processing and analysis, and perform a series of operations such as algorithm calculation, pattern recognition, and statistical analysis on the data set according to the preset or dynamically generated data analysis logic;

[0029] 2.8 Generate analysis results. After the plug-in is called, the analysis data is input into the large language model. Based on the semantic matching results, feedback information is generated to inform the user that the system has understood its request and assigned the corresponding plug-in. After the large model returns the results, it is displayed in the form specified by the user, such as charts, reports, interactive controls or other visualization methods, allowing users to provide feedback on the results of semantic analysis. The system makes adjustments based on the feedback to improve the accuracy of semantic understanding.

[0030] Furthermore, the interfaces provided by the standardized plug-in platform include but are not limited to data input and output, data processing requests, result feedback, etc.

[0031] Furthermore, the step of searching or customizing a plug-in in the plug-in library includes searching, selecting, configuring and installing the plug-in by the user through a user interface.

[0032] Furthermore, the plug-in library supports version control and dependency management to facilitate plug-in updates and maintenance.

[0033] Furthermore, the plug-in library provides information such as functions, ratings, and user feedback to assist users in making choices.

[0034] Furthermore, the step of maintaining the dialog context includes using session management technology to record user interaction history and current status.

[0035] Furthermore, the natural language processing technology includes intent recognition, entity extraction and semantic understanding.

[0036] Furthermore, the natural language processing technology can dynamically adjust the semantic matching algorithm and data analysis logic according to the analysis results of the plug-in and the feedback information of the user to optimize the subsequent semantic analysis results.

[0037] Furthermore, the data analysis control center has task scheduling, performance monitoring and error handling capabilities to ensure the stability and reliability of the data analysis process.

[0038] Furthermore, the analysis result display supports custom formats and styles, allowing users to adjust according to personal preferences and needs.

[0039] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-mentioned customized plug-in driven data analysis method.

[0040] The present invention also provides a data analysis system, comprising a module for executing the above-mentioned customized plug-in driven data analysis method.

[0041] The present invention also provides a customized plug-in driven data analysis system, comprising:

[0042] 3.1 Natural language input module, receiving the requirements to be analyzed provided by the user;

[0043] 3.2 Data analysis module, responsible for receiving analysis requirements and integrating them with the big model, receiving the results returned by the plug-in calling the data analysis platform, processing and feeding back the analysis results to the display module;

[0044] 3.3 Language model module, receives natural language for semantic analysis, performs semantic matching on customized plug-ins, and feeds back processing results;

[0045] 3.4 Data management module, responsible for data access, preprocessing, storage, calculation and display, providing operating environment and support services for plug-ins, and providing standardized data interfaces for plug-ins to use;

[0046] 3.5 Plug-in library module, which stores various data analysis plug-ins that can be selected or customized by users, and stores various versions of plug-ins;

[0047] 3.6 Plug-in management module, which defines standardized plug-in interface specifications, ensures the compatibility and interoperability of plug-ins with the data analysis basic platform, and provides a user-friendly interface for plug-in selection, installation, uninstallation, configuration and other operations;

[0048] 3.7 Result display module, which displays data analysis results in an intuitive and easy-to-understand way according to the selected plug-in category, designs interactive charts, and supports result export for offline analysis and generation of professional reports.

[0049] Furthermore, the natural language input module includes speech recognition and text parsing functions, which can convert the user's speech or text input into structured analysis requirements.

[0050] Furthermore, the data analysis module includes a data analysis engine that is capable of performing multi-threaded processing and distributed computing.

[0051] Furthermore, the language model module includes intent recognition and entity extraction mechanisms, which can understand and execute complex language instructions to extract key information from users' natural language needs.

[0052] Furthermore, the data management module includes data access, data cleaning and conversion tools to ensure data quality and consistency and provide data security protection, including data encryption, access control and audit logs.

[0053] Furthermore, the plug-in library module includes a version control system, allowing users to roll back to an old version of a plug-in.

[0054] Furthermore, the plug-in management module allows users to upload custom plug-ins, perform version control on them, and provide a plug-in dependency check function to ensure the correct installation and operation of the plug-in.

[0055] Furthermore, the result display module includes a custom report generation tool, allowing the user to customize the format and content of the report according to specific needs.

[0056] The present invention also provides a computer system for executing the customized plug-in driven data analysis method described above, characterized in that the system includes a storage medium, at least one processor and a memory communicatively connected to the processor, the storage medium stores a computer program, and the computer program implements the steps of the above-mentioned data analysis system when executed by the processor.

[0057] According to another aspect of the present disclosure, a storage medium is provided, storing instructions, a data analysis result storage area for storing data analysis results, and a plug-in information storage area for storing information of a data analysis plug-in. When these instructions are read and executed by a computer system, the computer system is enabled to execute the above-mentioned customized plug-in-driven data analysis method.

[0058] The present invention also provides a data analysis method for realizing the functions of the customized plug-in driven data analysis system described above.

[0059] The beneficial effects of the present invention are: providing a customized plug-in driven data analysis method, system and storage medium, allowing users to perform personalized data analysis tasks by configuring specific plug-ins. These plug-ins follow a unified interface standard and can run independently of the main program, thereby achieving rapid expansion and customization of data analysis functions. Compared with the prior art, the present invention enhances the flexibility of data analysis and simplifies the complexity of user use. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The accompanying drawings are provided to better understand the present solution and more clearly illustrate the technical solution in the embodiments of the present invention, wherein:

[0061] Figure 1 is a flowchart illustrating a data analysis plug-in development process provided in an embodiment of the present invention;

[0062] Figure 2 It is a data analysis plug-in configuration manifest file (manifest.json) format provided in an embodiment of the present invention;

[0063] Figure 3 It is a flowchart of the execution sequence of a customized plug-in driven data analysis method provided in an embodiment of the present invention;

[0064] Figure 4It is a flowchart of a customized plug-in driven data analysis method provided in an embodiment of the present invention;

[0065] Figure 5 is another execution flow chart of a customized plug-in driven data analysis method provided in an embodiment of the present invention;

[0066] Figure 6 It is a functional schematic diagram of a customized plug-in driven data analysis system provided in an embodiment of the present invention.

[0067] Those skilled in the art should understand that the drawings are for illustration and explanation only and should not limit the specific implementation of the present invention. The specific structures and implementation methods of the components and steps in the drawings will be adjusted and optimized accordingly according to specific business scenarios and technical requirements. DETAILED DESCRIPTION

[0068] In a specific embodiment of the present invention, the implementation details of the customized plug-in driven data analysis method, system and storage medium will be described in detail. The following embodiments are only used to illustrate the present invention and are not intended to limit its scope.

[0069] This embodiment provides a plug-in definition method, including the following steps:

[0070] S101. Plug-in requirement analysis: clarify the data analysis problems that the plug-in is designed to solve, such as the need for a plug-in that can analyze e-commerce operation activities. The plug-in can output detailed advertising data, user landing page access data, etc., and can intuitively analyze activity exposure effects, new user activation conversions, old user invitation conversions, real-time new user acquisition effects, etc.

[0071] S102. Plug-in design: Design the internal structure of the plug-in, including the data input and output interface of e-commerce operation activities, processing flow, interaction method with the main program, etc. Form a design document to ensure that the plug-in design meets user needs. During the design stage, the scalability and maintainability of the plug-in should be considered.

[0072] S103. Plug-in interface specification formulation: Use OpenAPI (the specification of the interface definition language) to define the standardized interface for communication between plug-ins and the main program or other plug-ins. OpenAPI allows developers to define a complete blueprint of a RESTful API (representational state transfer application programming interface), which uses the JSON (JavaScript Object Notation) format to describe the API (application programming interface), and can define information such as requests, responses, and parameters.

[0073] S1031. Create an OpenAPI document to describe the basic properties of the plugin, such as information, version, and server address.

[0074] S1032. Define API endpoints, including paths, supported HTTP methods (such as GET, POST), request parameters, and request bodies.

[0075] S1033. Define a response message for each API, including a status code, a response body structure, and possible error information.

[0076] S1034. Use OpenAPI UI to generate API documentation so that developers and users can view and test the API.

[0077] S104. Plug-in interface development and programming implementation:

[0078] S1041. Based on the design document and OpenAPI definition, select an appropriate programming language (such as Python, Java, or C#) and development environment for coding.

[0079] S1042. Implement the API interface defined by OpenAPI, develop query event analysis logic, ensure that the interface is coded correctly, and can receive external requests and return expected responses.

[0080] S1043. Write an exception handling module for the plug-in to ensure that reasonable feedback can be given when encountering incorrect input or processing errors.

[0081] S1044. Perform unit testing to verify whether the algorithm logic of each interface of the plug-in works correctly.

[0082] S1045. Conduct integration testing to ensure that the e-commerce operation activity data interface parameters are input correctly and the result response is correct, and ensure that the plug-in runs smoothly on the data analysis basic platform.

[0083] S105.Test verification: Perform comprehensive testing on the plug-in, including unit testing, functional testing, and performance testing, to ensure that the plug-in works as expected and interacts and executes functions correctly in the main program.

[0084] S106. Plug-in release: Plug-ins that have passed the test will be released to the plug-in library in the plug-in management center, and detailed user manuals, installation guides and API documents will be provided so that developers and users can understand how to correctly install, configure and use the plug-ins.

[0085] S1061. Add a new plug-in named E-commerce Event Analysis in the plug-in management platform and upload the manifest.json file. The manifest.json file is a configuration manifest file in JSON format and fills in the plug-in function and usage scenario description.

[0086] The S1062.manifest.json file configuration manifest file includes schema_version (plug-in version), name_for_human (display name), name_for_model (input model name), description_for_human (display description), description_for_model (input model description), api.type (API type), api.url (open API configuration file openapi.yaml address), logo_url (LOGO address), etc.

[0087] S1063. The plug-in management platform parses the manifest.json configuration manifest file, pulls the API configuration file according to the openapi.yaml address, parses it into a large model function description file, and stores it in the plug-in library.

[0088] S107. Maintenance and Updates: After the plugin is released, we will provide regular technical support, maintain the plugin version, fix errors found in the program, respond to user feedback, and improve performance and functionality. This ensures the long-term stability and availability of the plugin.

[0089] The above steps can quickly and efficiently define a plug-in, such as Figure 1 The plug-in development process can ensure that the entire process from plug-in concept to implementation is standardized and systematic, such as Figure 2 The file configuration format of the plugin list is given to more clearly describe the plugin version, purpose, and OpenAPI file location path. The use of OpenAPI provides clear API documentation and specifications for the plugin, simplifying the integration and testing process of the plugin.

[0090] like Figure 3 As shown, this embodiment provides a customized plug-in driven data analysis method, comprising the following steps:

[0091] S201. Build a data analysis basic platform, which includes but is not limited to the following steps:

[0092] S2011. Data source access enables access to multi-source heterogeneous data and supports data reporting from multiple channels such as browsers, mobile apps, applets, and backend servers.

[0093] Provides API interfaces for data reporting, including RESTful API and message queues (such as Kafka, RabbitMQ), etc.

[0094] Ensure the real-time and reliability of data reporting, and adopt efficient data compression and transmission encryption technology.

[0095] Design a flexible data reporting protocol that supports custom data formats and structures to adapt to the characteristics of different data sources.

[0096] S2012. Data preprocessing is responsible for cleaning the reported raw data, not limited to recording event attributes, user attributes, and virtual attributes of the reported data, including removing duplicate records, correcting erroneous information, and filling in missing values.

[0097] Perform data transformation to convert data into a unified format for subsequent processing and analysis.

[0098] Perform data standardization, including unit conversion, numerical normalization, etc., to eliminate differences between different data sources.

[0099] Use streaming computing engines (such as Apache Flink or Spark Streaming) to process real-time data and quickly generate queryable data views.

[0100] S2013. Data storage, using distributed file systems (such as Hadoop HDFS) or real-time analytical databases (such as Doris) to store large data sets.

[0101] Ensure high availability and disaster recovery capabilities of data storage through data backup and replication mechanisms.

[0102] Supports efficient access to data and uses indexing, partitioning, caching and other technologies to improve query speed.

[0103] Realize secure data storage, including data encryption, access control, audit log and other security mechanisms.

[0104] S2014. Data analysis and computing, integrating multiple data analysis algorithms, such as regression analysis, cluster analysis, classification analysis, etc., as well as machine learning models, such as decision trees, neural networks, etc.

[0105] Supports user-defined analysis algorithms and provides algorithm development tools and API interfaces.

[0106] Implement parallel computing and distributed computing to improve the processing speed and scalability of analysis tasks.

[0107] Provides a caching mechanism for data analysis results to improve the efficiency of repeated queries.

[0108] S2015.Results display, providing data visualization tools, supporting data display in various forms such as charts, maps, and dashboards.

[0109] Allow users to customize visualization components, including selecting different data dimensions, visualization types, and styles.

[0110] Provides a user interaction interface, including data exploration, report generation, data sharing and other functions.

[0111] It supports interactive data analysis, and users can perform operations such as data filtering, drilling, and correlation analysis through the interface.

[0112] S202. Design and develop a standardized plug-in platform that defines a clear set of API interface specifications and a plug-in development framework. Use the OpenAPI (formerly Swagger) specification to define the plug-in API interface to ensure standardization and automated document generation of the interface. Define the request and response format of the plug-in, including JSON Schema (JSON metadata information) to describe the data structure. Design API endpoints, including data input, processing, output, and status checking.

[0113] The platform provides a set of common code templates and development tools to simplify the plug-in development process. The framework should include common functions such as data access, exception handling, and logging. It also provides plug-in development guidelines and best practice documents to help developers get started quickly.

[0114] Third-party developers are allowed to write customized plug-ins according to specific data analysis requirements. The customized plug-in process is a plug-in definition method described in the previous embodiment S1.

[0115] The plug-in should contain core logic such as data processing algorithms, statistical models, machine learning models, follow the platform's interface specifications to ensure compatibility, and seamlessly connect with the data analysis basic platform through the interface.

[0116] S203. Users can easily browse the plug-in library through a graphical user interface (GUI) and select appropriate plug-ins based on functions, ratings, user feedback and other information.

[0117] Provides a search function that allows users to search for specific plugins based on keywords. Implements a filtering mechanism that allows users to filter plugins based on data type, analysis type, algorithm complexity, and other criteria.

[0118] Allow users to view detailed information about the plug-in, including functional descriptions, performance indicators, user reviews, use cases, etc. Once the user selects a plug-in, the platform will carry the plug-in for subsequent analysis needs.

[0119] S204. Users submit their data analysis requirements through the system's graphical user interface (GUI), which can be described in natural language, such as "analyze the new user conversion of e-commerce operations in the past month", and select multiple plug-ins browsed in the plug-in library to submit analysis requirements.

[0120] S205. The system uses natural language processing technology to parse the user's natural language needs, extract key information, such as analysis objectives, time range, data type, etc., and convert user needs into structured queries.

[0121] The system uses natural language processing technology to semantically match the user's intention with the custom plug-ins in the plug-in library to determine whether a specific plug-in or a series of plug-ins needs to be executed to meet the needs, such as Figure 4 The flowchart executed by the unmatched plug-in is as follows: Figure 5 Flowchart of the steps executed to match a plug-in. If the requirements are clear and a matching plug-in is available, the system sends the matching plug-in to the data analysis control center. If the requirements are complex or non-standard, the system may need to invoke a custom plug-in development process or request further information from the user.

[0122] S206. The data analysis control center calls the plug-in to query the data set from the data platform. The data set processed by the plug-in is summarized and integrated into a unified format so that it can be input into the large language model for analysis. The integrated data set is input into the large language model, which may include a machine learning model, a deep learning network or other advanced analysis tools.

[0123] S207. Generate analysis results. After the plug-in is called, the summarized data set is input into the large language model. The large language model performs in-depth analysis on the data, which may include complex pattern recognition, predictive analysis, trend analysis, etc., to facilitate display and user understanding, such as data simplification, summary generation, data aggregation, etc., to generate feedback information.

[0124] S208. Display of data analysis results: converting the data analysis results into a visual format, such as charts (bar charts, line charts, pie charts), scatter plots, heat maps, tree diagrams, etc.

[0125] The display style and layout can be customized according to the plug-in category selected by the user and personal preferences, and data can be displayed from multiple dimensions, allowing users to view analysis results from different angles as needed.

[0126] Allows users to export results to different formats, such as CSV, Excel, PDF, etc., for offline analysis or report making.

[0127] The performance of the result display module is optimized to ensure a smooth user experience even when processing large amounts of data.

[0128] Design a user feedback mechanism so that users can provide feedback on the displayed results, and the system will make adjustments based on the feedback to provide more accurate analysis results.

[0129] This embodiment provides a customized plug-in driven data analysis system, including the following components:

[0130] S301. Natural language input module, design an intuitive user interface that includes a text input area or voice input button, allowing users to express their data analysis needs in the form of natural language.

[0131] For voice input devices, integrate advanced speech recognition technologies, such as Google Speech-to-Text or Amazon Transcribe or iFlytek’s voice dictation, to convert the user’s voice into text.

[0132] The system automatically detects the language of the user's input and selects the corresponding language model for processing. Using contextual understanding technology, the system is able to understand the user's intentions and needs, even if the user's input may be ambiguous or incomplete. The text entered by the user is preprocessed, including removing punctuation, converting to lowercase, word segmentation, etc., in preparation for language processing.

[0133] Use intent recognition algorithms, such as machine learning or deep learning models, to determine which data analysis needs the user's input belongs to. Extract key entities and attributes from the user's input, such as time range, data type, specific statistical requirements, etc.

[0134] If necessary, the system can start a dialogue process to collect more information through interaction with the user to ensure that the user's needs are fully understood. The user's input and the system's response are recorded for subsequent analysis and optimization.

[0135] S302. The data analysis module is the core of the data analysis system and is responsible for coordinating and managing the entire analysis process, such as Figure 5 It is a flowchart of plugin calls.

[0136] The data analysis module receives the user's analysis requirements through the natural language input module and converts the user's requirements into a format that the system can understand. It analyzes the user's requirements in detail, extracts key information, and integrates the corresponding plug-ins in the plug-in library based on the requirements analysis results.

[0137] Call the language model module, perform semantic matching based on the user-specified plug-in and input analysis requirements, and obtain the plug-in semantic matching results.

[0138] Call the data infrastructure platform according to the matching plug-in parameters, obtain the data set returned by the plug-in, and perform necessary post-processing, such as data aggregation, outlier processing, etc.

[0139] The data set is then input into the data domain language model for analysis and processing to obtain feedback results, and the analysis feedback results of the model are passed to the result display module so that users can view and interact.

[0140] When an exception is encountered during the analysis process, the system can handle the error and give a user-friendly error prompt. Record key events during the analysis process, including user needs, operations performed, errors encountered, and system responses. Provide a user interaction mechanism to allow users to provide feedback or intervene during the analysis process.

[0141] S303. The language model module is a key component in the data analysis system, which is responsible for understanding and processing natural language input, as well as semantic matching with custom plug-ins.

[0142] This module receives natural language text submitted by the user through the natural language input module.

[0143] Preprocess the input natural language text, including removing sensitive words, stop words, stemming, part-of-speech tagging, etc., in preparation for semantic analysis. Use natural language processing (NLP) techniques, such as dependency parsing and entity recognition, to conduct in-depth semantic analysis of the text.

[0144] Determine the user's input intent, for example, whether they are requesting data analysis, specific report generation, or data visualization.

[0145] Semantically match the user's intent and the extracted entities with the custom plugins in the plugin library to find the most suitable plugin to execute the user's request.

[0146] Maintain conversational context for consistency and accuracy across multiple turns of interaction.

[0147] Based on the semantic matching results, feedback information is generated to inform the user that the system has understood their request and assigned the corresponding plug-in. If an error occurs during semantic analysis or plug-in matching, clear error feedback is provided and the user is asked to provide more information if possible. Users are allowed to provide feedback on the results of semantic analysis, and the system makes adjustments based on the feedback to improve the accuracy of semantic understanding.

[0148] Ensure compliance with data protection regulations when processing natural language input, protecting the security and privacy of user data.

[0149] S304. The data management module is the core component of the data analysis system. It is responsible for all links in the data life cycle and ensures the accuracy, integrity and security of the data.

[0150] Implement multiple data access methods, including database connection, file import, API integration, etc., to support data access and reporting from different sources.

[0151] Perform pre-processing operations such as cleaning, processing, and format conversion on the received data to ensure data quality.

[0152] Design efficient data storage solutions to support the storage of structured and unstructured data. Implement data indexing and partitioning to improve data retrieval and query efficiency, and implement fault-tolerant mechanisms for data management modules, such as data backup and failover, to ensure high availability of the system.

[0153] Provides a data computing engine that supports various data analysis and processing algorithms. Supports distributed computing to process large-scale data sets. Implements data consistency control mechanisms, such as transaction management, concurrency control, etc., to ensure data consistency under concurrent operations. Optimizes the performance of the data management module, including query optimization, cache strategy, load balancing, etc., to improve system response speed.

[0154] Provides standardized data access interfaces for plug-ins, including data query, data update, data deletion and other operations.

[0155] S305. The plug-in library module is an important component of the data analysis system. As a centralized storage and distribution center, it implements a centralized plug-in storage system for storing various data analysis plug-ins, including pre-made plug-ins and user-defined plug-ins.

[0156] Categorize and index plugins so that users can quickly find the plugins they need based on data analysis type, function, complexity, etc. Provide a search function that allows users to search for specific plugins by keywords. Implement a filtering mechanism so that users can filter plugins based on data type, analysis requirements, user reviews, etc.

[0157] Provide a details page for each plug-in to display the plug-in's function description, use cases, user reviews, version information, etc.

[0158] Implement a version control mechanism to store different versions of plugins.

[0159] S306. The plug-in management module is a key part of the data analysis system. It is responsible for maintaining the life cycle of the plug-in, ensuring the compatibility of the plug-in, and providing a user interface to facilitate users to perform various plug-in related operations.

[0160] Develop a user-friendly graphical user interface (GUI) that enables users to easily select, install, uninstall, and configure plugins. Provide an installation wizard to guide users through the plugin installation process, including dependency checking, permission configuration, configuration list checking, OpenAPI file checking, etc.

[0161] Develop a set of standardized plug-in interface specifications, including data exchange formats, API specifications, event handling mechanisms, etc., to ensure that the plug-in can be seamlessly integrated with the data analysis infrastructure platform.

[0162] Implement version control functionality to allow users to view, install, or roll back to a specific version of a plugin.

[0163] Provide an uninstall function to allow users to uninstall plug-ins that are no longer needed and ensure that all related files and configurations are cleared during the uninstall process.

[0164] Implement a compatibility testing process to ensure that newly developed or updated plug-ins can work properly with the current version of the data analysis infrastructure platform, ensure the permissions and security of the plug-in installation and runtime, and prevent malicious plug-ins from causing damage to the system.

[0165] If applicable, integrate a plugin marketplace to allow users to browse, search, and install plugins provided by third-party developers.

[0166] Monitor the performance of plugins to ensure that they do not negatively impact the overall performance of the system.

[0167] S307. The result display module is the view module of the data analysis system, responsible for displaying the results of the data analysis in a user-friendly manner.

[0168] Receive processing results from the data analysis module, convert the data analysis results into visual formats, such as charts (bar charts, line charts, pie charts), scatter plots, heat maps, tree diagrams, etc., and receive professional data analysis reports from the data analysis module.

[0169] Design interactive charts that allow users to interact with data by clicking, dragging, zooming, and more to explore the data in depth.

[0170] The display style and layout can be customized according to the plug-in category selected by the user and personal preferences, and data can be displayed from multiple dimensions, allowing users to view analysis results from different angles as needed.

[0171] Real-time data updates ensure that users always see the latest analysis results. Allow users to export results to different formats, such as CSV, Excel, PDF, etc., for offline analysis or report making.

[0172] The performance of the result display module is optimized to ensure a smooth user experience even when processing large amounts of data.

[0173] Design a user feedback mechanism to allow users to make suggestions for improving the visualization results.

[0174] This embodiment provides a storage medium, characterized in that the following instructions and data are stored:

[0175] S401. Data analysis instruction set, used to guide the computer system to execute the data analysis process, including data preprocessing, model training, result verification and other steps.

[0176] S402. The data analysis result storage area is used to store the intermediate results and final results generated during the analysis process, and adopts an efficient data organization and indexing mechanism to support fast data retrieval and access.

[0177] S403. A plug-in information storage area is used to store metadata, configuration parameters, version information, etc. of the data analysis plug-in to facilitate plug-in management and version control of the system. When these instructions are read and executed by the computer system, the computer system can execute the above-mentioned customized plug-in driven data analysis method. The storage medium can be a hard disk drive (HDD), a solid state drive (SSD), an optical disk, a USB flash drive, a memory card, etc., or a network storage device such as a network attached storage (NAS) or a storage area network (SAN).

[0178] In the embodiments of the present invention, by providing a customized plug-in driven data analysis method, system and storage medium, a high degree of customization and flexibility of data analysis tasks is achieved. Users can select or develop specific plug-ins to expand data analysis functions according to their needs without having to deeply understand the underlying technical details. In addition, the modular design and standardized interface specifications of the system also greatly improve the maintainability and scalability of the system.

[0179] The above description is only a preferred embodiment of the present invention. For those skilled in the art, many changes and modifications can be made based on the above content without departing from the idea or essence of the invention, which should all fall within the scope of protection of the present invention.

Claims

1. A plug-in definition method, characterized in that: include: 1.1 Plug-in requirements analysis: identify and clarify the purpose and function of the plug-in, determine the specific problems that the plug-in should solve, the expected behavior and performance requirements; 1.2 Plug-in design, create a plug-in architecture and design specification list, clearly define the plug-in's internal structure, interface definition, data flow, and interaction with the main program or other plug-ins; 1.3 Develop plug-in interface specifications to define the standardized interface for communication between plug-ins and main programs or other plug-ins, including input and output parameters, calling protocols, and data formats; 1.4 Programming implementation: coding of plug-ins is carried out according to design documents and interface specifications, and appropriate programming languages ​​and tools are used to develop executable plug-in components; 1.5 Test verification, implement unit testing, functional testing and performance testing to ensure that the plug-in works as expected and interacts and executes functions correctly in the main program; 1.6 Plug-in release: publish the compiled plug-in list to the plug-in library, provide detailed user manuals, installation guides and API documentation, and allow users to rate and provide feedback on the plug-in; 1.7 Maintenance updates, regular technical support, maintenance of plug-in versions, patches for bugs found in the program, response to user feedback, and performance and functionality improvements.

2. The plug-in definition method according to claim 1, characterized in that: The design specification list in step 1.2 plug-in design step includes a detailed description of the plug-in's extensibility, security, and user customization options.

3. The plug-in definition method according to claim 1, characterized in that: In the step 1.3 of formulating the plug-in interface specification, the interface specification is defined using the OpenAPI specification.

4. A customized plug-in driven data analysis system, characterized in that: include: 4.1 Natural language input module, used to receive the requirements to be analyzed provided by the user; 4.2 Data analysis module, responsible for receiving analysis requirements and integrating with the big model, receiving the results returned by the plug-in calling the data analysis basic platform, processing and feeding back the analysis results to the display module; 4.3 Language model module, used to receive natural language for semantic analysis, semantically match customized plug-ins, and feedback processing results; 4.4 Data management module, responsible for data access, preprocessing, storage, calculation and display functions, providing operating environment and support services for plug-ins, and providing standardized data interfaces for plug-ins to use; 4.5 Plug-in library module, used to store various data analysis plug-ins that can be selected or customized by users, and store various versions of plug-ins; 4.6 Plug-in management module, which is used to define standardized plug-in interface specifications, ensure the compatibility and interoperability of plug-ins with the data analysis basic platform, and provide a user-friendly interface for plug-in selection, installation, uninstallation, and configuration operations; 4.7 Result display module is used to display data analysis results in an intuitive and easy-to-understand way according to the selected plug-in category, and to design interactive charts, support result export, facilitate offline analysis and generate professional reports.

5. The customized plug-in driven data analysis system according to claim 4, characterized in that: The natural language input module includes speech recognition and text parsing functions, and can convert the user's speech or text input into structured analysis requirements.

6. The customized plug-in driven data analysis system according to claim 4, characterized in that: The language model module includes intent recognition and entity extraction mechanisms, which can understand and execute complex language instructions to extract key information from users' natural language needs.

7. The customized plug-in driven data analysis system according to claim 4, characterized in that: The plug-in management module allows users to upload custom plug-ins, allows users to roll back to old versions of plug-ins, performs version control on them, and provides a plug-in dependency check function to ensure the correct installation and operation of plug-ins.

8. A computer system for executing the customized plug-in driven data analysis method according to any one of claims 1 to 7, characterized in that: The system includes a storage medium, at least one processor and a memory connected to the processor in communication, wherein the storage medium stores a computer program, and when the computer program is executed by the processor, the steps of the above data analysis system are implemented.

9. A computer-readable storage medium, characterized in that: A computer program is stored, and when the computer program is executed by a processor, the steps of the customized plug-in driven data analysis system according to any one of claims 1 to 7 are implemented.

Images