Data analysis method and device, equipment, storage medium and computer program product

Abstract

The invention provides a data analysis method and device, equipment, a storage medium and a computer program product, and relates to the technical field of computers, the method comprises the following steps: obtaining target data related to a specific field, and extracting core data associated with a data analysis target set by a user from the target data; performing hierarchical progressive analysis on the core data to obtain a data analysis result; and interpreting and sorting the data analysis result to obtain insight data of the data analysis result, and outputting the insight data. According to the method, useful data are extracted from large-scale and disordered data through hierarchical progressive analysis and insight analysis, so that a relationship, correlation and insight hidden in the chaotic data are obtained, and a data analysis target set by a user is realized.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a data analysis method, device, equipment, storage medium and computer program product. Background Art

[0002] With the rapid development of science and technology, a large amount of data is generated in various fields. The amount of this data is growing exponentially, especially in the fields of scientific research, finance, healthcare, and social media, and it is becoming increasingly difficult to extract useful information from this data.

[0003] Traditional data analysis methods rely on manual processing and analysis, which is not only inefficient but also difficult to deal with such large-scale data sets. In addition, these data sets contain a lot of noise and unstructured information, which further increases the difficulty of data analysis. Therefore, how to extract useful data from large-scale and chaotic data has become an urgent problem to be solved. Summary of the invention

[0004] The present invention provides a data analysis method, apparatus, device, storage medium and computer program product to solve the defects in the prior art of how to extract useful data from large-scale and chaotic data. Through hierarchical progressive analysis and insight analysis, useful data can be extracted from large-scale and chaotic data, thereby obtaining the hidden relationships, correlations and insights in the chaotic data, and achieving the data analysis goals set by the user.

[0005] The present invention provides a data analysis method, comprising the following steps: Acquire target data related to a specific field, and extract core data associated with the data analysis target set by the user from the target data; Performing hierarchical progressive analysis on the core data to obtain data analysis results; The data analysis results are interpreted and sorted to obtain insight data of the data analysis results, and the insight data is output.

[0006] According to a data analysis method provided by the present invention, the step of performing hierarchical progressive analysis on the core data to obtain data analysis results includes: Using a simple mode, basic data analysis is performed on the core data to obtain first analysis data; If the first analysis data does not match the data required by the data analysis target, at least one machine learning model in the intermediate mode is used to perform data analysis on the core data to obtain second analysis data; If the second analysis data does not match the data required by the data analysis target, at least one deep learning model in the advanced mode is used to perform data analysis on the core data to obtain the data analysis result.

[0007] According to a data analysis method provided by the present invention, after acquiring target data related to a specific field and extracting core data associated with the data analysis target set by the user from the target data, the method further includes: Determine the complexity of the target data and the required depth of insight; the depth of insight represents the degree of multi-level interpretation of the data; Determining at least one analysis mode according to the complexity, the insight depth, and the data analysis goal; According to at least one of the analysis modes, data analysis is performed on the core data to obtain the data analysis result.

[0008] According to a data analysis method provided by the present invention, the specific field includes at least the field of scientific research, the field of finance, the field of health care, and the field of social media; the acquisition of target data related to the specific field includes: Acquire numerical data and text data related to the specific field from at least one data source; the text data is used to describe and annotate the numerical data; The numerical data and the text data are preprocessed to obtain target data related to the specific field.

[0009] According to a data analysis method provided by the present invention, the data analysis results are interpreted and sorted to obtain insight data of the data analysis results, including: Conducting an in-depth interpretation of the data analysis results to identify key data in the data analysis results; The key data is converted into the insight data.

[0010] According to a data analysis method provided by the present invention, the data analysis results are interpreted and sorted to obtain insight data of the data analysis results, and after outputting the insight data, the method comprises: Obtaining user feedback data on the insight data; According to the feedback data, the data analysis model or model parameters are adjusted; the data analysis model includes a machine learning model and a deep learning model.

[0011] The present invention also provides a data analysis device, comprising the following modules: A core data extraction module is used to obtain target data related to a specific field and extract core data associated with the data analysis target set by the user from the target data; A data analysis module is used to perform hierarchical progressive analysis on the core data to obtain data analysis results; The insight data output module is used to interpret and organize the data analysis results, obtain insight data of the data analysis results, and output the insight data.

[0012] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above-mentioned data analysis methods when executing the computer program.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the data analysis method described in any one of the above is implemented.

[0014] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the data analysis methods described above.

[0015] The data analysis method, device, equipment, storage medium and computer program product provided by the present invention obtain target data related to a specific field, extract core data associated with the data analysis target set by the user from the target data; perform hierarchical progressive analysis on the core data to obtain data analysis results; interpret and organize the data analysis results to obtain insight data of the data analysis results, and output the insight data. The present invention extracts useful data from large-scale and chaotic data through hierarchical progressive analysis and insight analysis, thereby obtaining hidden relationships, correlations and insights in chaotic data, and achieving the data analysis target set by the user. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 It is a flow chart of the data analysis method provided by the present invention.

[0018] Figure 2 It is a flow chart of the intelligent method for discovering knowledge from chaotic data using a large model provided by the present invention.

[0019] Figure 3 It is a structural schematic diagram of the data analysis device provided by the present invention.

[0020] Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] Combine the following Figure 1-Figure 4 The data analysis method, apparatus, device, storage medium and computer program product of the present invention are described.

[0023] Figure 1 It is a flow chart of the data analysis method provided by the present invention, such as Figure 1 As shown, the method includes the following: Step 101: Acquire target data related to a specific field, and extract core data associated with a data analysis target set by a user from the target data.

[0024] It should be noted that the execution subject of the present invention is a data analysis system, specifically an intelligent system that uses a large model to discover knowledge from chaotic data. The system uses a large model to discover hidden relationships, correlations and insights from a large amount of chaotic data. The system implements data analysis through a data input module, a goal setting module, a pattern recognition module, an insight generation module, a visualization module and a learning and feedback module.

[0025] Specific fields include at least scientific research, finance, healthcare, and social media. Target data related to specific fields include numerical data and text data, where numerical data can include various financial indicators, experimental results, market data, etc., while text data is mainly used to explain and annotate relevant numerical data, such as data sources, measurement conditions, or background information of data sets.

[0026] The system uses the goal setting module to extract core data associated with the data analysis goals set by the user from the target data. Among them, the goal setting module is one of the core parts of the system, which aims to help users set clear analysis directions and achieve specific goals in a data-driven way. Users can set specific goals according to the needs of their fields. For example, in the financial field, users can set goals such as "increasing profits", "reducing risks" or "optimizing investment portfolios"; in the field of scientific research, users can set goals such as "increasing the success rate of experiments" or "discovering new related factors". The system will guide the entire process of data analysis around the data analysis goals set by users.

[0027] Once the user sets the data analysis goal, the system will first automatically identify the core data that is closely related to the data analysis goal. The core data can be understood as numerical and text information with important value and relevance after being processed in the data input module. The core data can support effective pattern recognition and insight generation, thereby providing support for decision-making. For example, if the data analysis goal set by the user is "increase profits", the system will identify key data points such as sales, costs, market trends, and customer behavior data. Further, the core data is preliminarily processed, such as screening, merging, and standardization, to ensure the accuracy and consistency of the data. This process can help the system establish a preliminary relationship model between the data analysis goals and these core data.

[0028] Step 102: Perform hierarchical progressive analysis on the core data to obtain data analysis results.

[0029] The system uses a pattern recognition module to perform hierarchical and progressive analysis on the core data to obtain data analysis results. Among them, the pattern recognition module is one of the core components of the system, responsible for mining valuable patterns and laws from the data. This module adopts a hierarchical and progressive data analysis method, which is divided into simple mode, intermediate mode and advanced mode, which are suitable for data analysis needs of different complexity. Users can freely choose the appropriate analysis mode according to specific analysis goals and needs. In addition, the system has preset multiple models, and can repeatedly iterate and optimize the analysis results according to the user's choice and the characteristics of the data to ensure that the final analysis results can achieve the user's goals to the greatest extent.

[0030] The specific analysis of the simple mode, intermediate mode and advanced mode is as follows: (1) Simple mode aims to quickly identify potentially useful patterns through basic analysis of data and simple conditional combinations. This mode is suitable for users who want to obtain preliminary analysis results in a shorter period of time or when the complexity of the data is low. For example, when processing financial data, the system can quickly identify the factors that most directly affect profit growth through simple conditional combinations, such as cross-analysis of sales, costs, and market trends. The system can also set iteration thresholds in simple mode to ensure the efficiency of the analysis process. For example, the system can set a limit of up to 5 combinations of data, and users can also adjust the number of combinations as needed. If the user wants to explore the combination effect of a specific data with other data, the system can fix the data as the core variable of the analysis, thereby finding the optimal combination within the preset combination range.

[0031] (2) The intermediate mode adds more complex machine learning algorithms to the simple mode to identify deeper patterns and relationships in the data. This mode is suitable for situations where the amount of data is large or the data relationships are complex, and users hope to obtain more accurate results through more advanced analysis methods. In the intermediate mode, the system will use a variety of algorithms including supervised learning and unsupervised learning. For example: clustering algorithms (such as K-means, DBSCAN): used to group similar data points, reveal hidden structures in the data, and help users understand the internal relationships of the data; decision trees and random forests: used to classify and regress data, helping users identify variables that have a significant impact on the target variable; support vector machines (SVM): used to handle classification problems of linear and nonlinear data, helping users identify complex decision boundaries. The system also allows users to customize model parameters and iteration thresholds according to analysis needs. For example, users can choose to increase the number of iterations or adjust the complexity of the model to obtain more refined analysis results.

[0032] (3) Advanced mode is the most complex and powerful analysis mode in the system, suitable for scenarios where data is deeply mined and complex patterns are identified. This mode mainly relies on deep learning technology, which can process high-dimensional and unstructured data and identify patterns that are difficult to detect with traditional analysis methods. In advanced mode, the system uses a variety of advanced algorithms including deep neural networks, such as: Convolutional Neural Networks (CNN): Although mainly used for image data processing, it can also be used in financial data to detect complex patterns in time series data; Recurrent Neural Networks (RNN) and Long Short-Term Memory (LSTM): Suitable for processing time series data, they can capture long-term dependencies and trend changes in data. The system also integrates anomaly detection capabilities to identify abnormal patterns that may indicate key insights or data errors by monitoring the outliers output by the model. Analysis in advanced mode usually proceeds through multiple rounds of iterations, and the system automatically adjusts the parameters and structure of the model according to the set thresholds to gradually optimize the analysis results.

[0033] Whether in simple mode, intermediate mode or advanced mode, the system supports user-defined analysis processes and model selection. Users can select the most suitable analysis mode and model according to actual needs, and further optimize the analysis process by adjusting parameters and setting iteration thresholds. The system also presets a large number of analysis models, which users can choose to use according to actual needs. If the user is particularly interested in the role of a specific data point, the system can fix that data point and find the best combination among other data points to achieve the user's analysis goal. The system will automatically perform multiple rounds of analysis and optimization according to the preset iteration threshold until the optimal analysis result is achieved.

[0034] In one embodiment, a simple mode is used to perform basic data analysis on the core data to obtain first analysis data; if the first analysis data does not match the data required by the data analysis target, at least one machine learning model in the intermediate mode is used to perform data analysis on the core data to obtain second analysis data; if the second analysis data does not match the data required by the data analysis target, at least one deep learning model in the advanced mode is used to perform data analysis on the core data to obtain data analysis results.

[0035] Basic data analysis can be understood as preliminary processing and simple statistical analysis of data to help understand the basic characteristics and trends of the data. For example, in simple mode, the system performs basic data analysis such as descriptive statistics (such as calculating mean, median, standard deviation and variance, etc.), correlation analysis, data cleaning, data aggregation, etc. on the core data to obtain the first analysis data. If the first analysis data meets the data required by the data analysis target set by the user, the first analysis data can be directly output as the data analysis result; if the first analysis data does not meet the data required by the data analysis target set by the user, a more detailed analysis of the core data is required. In intermediate mode, the core data is input into the machine learning model to obtain the second analysis data output by the machine learning model. If the second analysis data meets the data required by the data analysis target set by the user, the second analysis data can be directly output as the data analysis result; if the second analysis data does not meet the data required by the data analysis target set by the user, a more detailed analysis of the core data is required. In advanced mode, the core data is input into the deep learning model to obtain the data analysis result output by the deep learning model.

[0036] Through this progressive analysis method, the pattern recognition module of the embodiment of the present invention can effectively respond to data analysis needs of different complexities, helping users to gradually and deeply explore the hidden rules in the data from simple condition combinations to complex deep pattern recognition, thereby achieving the goals set by users.

[0037] Step 103: interpret and organize the data analysis results to obtain insight data of the data analysis results, and output the insight data.

[0038] The system interprets and organizes the data analysis results through the insight generation module. Among them, the insight generation module is the key link in the system to transform complex data analysis results into actionable insights. The main function of this module is to interpret and organize the original patterns and relationships discovered by the pattern recognition module, and finally generate insights and suggestions that have direct application value to users. These insights can be widely used in various fields, such as finance, scientific research and other industries, to help users make more informed decisions.

[0039] In one embodiment, the data analysis results are deeply interpreted to identify key data in the data analysis results and convert the key data into insight data, wherein the key data includes key indicators, trends, related information, etc.

[0040] For example, in the financial field, the insight generation module can generate predictive models for investment strategies based on the correlation between market indicators and stock prices. For example, the system may find that certain macroeconomic indicators, industry trends, or company financial data are significantly correlated with stock price fluctuations. Based on these findings, the system can generate a series of investment recommendations, such as when to buy or sell specific stocks, or build an optimized investment portfolio to maximize returns and reduce risks. In the field of scientific research, the insight generation module can propose new hypotheses or research directions based on the complex variable relationships in experimental data. Suppose that in a biological experiment, the system identifies an unexpected relationship between ambient temperature and experimental results. Based on this finding, the insight generation module can suggest that researchers further explore the role of temperature as an influencing factor and may propose new experimental designs to verify this hypothesis.

[0041] Another important function of the insight generation module is to provide multi-level insight output, which is ultimately ranked and presented to the user based on the degree of goal achievement. For different levels of users, the module can generate simple summary insights to detailed technical suggestions. For example, for senior managers, the module will generate concise strategic suggestions, while for data analysts or researchers, the module will provide detailed analysis reports, including data sources, analysis methods, model parameters, and derivation processes. In order to ensure the effectiveness and practicality of the insights, the insight generation module also supports a user feedback mechanism. Users can evaluate the generated insights, and the system will continuously optimize the insight generation process based on these feedbacks to ensure that future analysis results are more in line with user needs.

[0042] Through the insight generation module, the system can effectively bridge the gap between complex data analysis and practical applications, transforming the patterns and regularities hidden in the data into specific and actionable suggestions, helping users achieve their goals and create more value in various fields.

[0043] The insight generation module generates insight data and outputs insight data. For example, the insight data is displayed through the visualization module. The visualization module presents the insight data generated by the system in an intuitive way, ensuring that users can understand the insights more easily. Users can also explore the data in depth, adjust analysis parameters, and discover hidden patterns through an interactive interface. The system supports a variety of visualization techniques to adapt to different types of data and analysis needs. Even non-expert users can easily obtain valuable information through graphical displays.

[0044] The data analysis method provided by the embodiment of the present invention obtains target data related to a specific field, extracts core data associated with the data analysis target set by the user from the target data; performs hierarchical progressive analysis on the core data to obtain data analysis results; interprets and organizes the data analysis results to obtain insight data of the data analysis results, and outputs the insight data. The present invention extracts useful data from large-scale and chaotic data through hierarchical progressive analysis and insight analysis, thereby obtaining the relationships, correlations and insights hidden in the chaotic data, and achieving the data analysis target set by the user.

[0045] Based on the above embodiment, after acquiring target data related to a specific field and extracting core data associated with the data analysis target set by the user from the target data, the method further includes: Step 210, determining the complexity of the target data and the required depth of insight; the depth of insight represents the degree of multi-level interpretation of the data; Step 211, determining at least one analysis mode according to the complexity, the insight depth and the data analysis target; Step 212: Perform data analysis on the core data according to at least one of the analysis modes to obtain the data analysis result.

[0046] In data analysis, the choice of simple mode, intermediate mode, and advanced mode depends on the complexity of the data, the data analysis goals, and the depth of insight required. The complexity of the data can include the following aspects: Data volume: The size and dimension of the data affect the complexity of the analysis. Large data sets may require more complex processing and analysis techniques. Data type: Data can be quantitative (numeric) or qualitative (categorical). Different types of data will affect the selected analysis method. Data structure: Whether the data has a hierarchical structure (such as time series, spatial data) or an association relationship (such as network data), etc., will also increase complexity.

[0047] The required depth of insight refers to the degree of multi-level interpretation of data during the analysis process, from simple descriptive information to complex cause-and-effect analysis, which can include the following aspects: Level of information: The depth of insight can be basic descriptive statistics or in-depth cause-and-effect analysis. The interactive relationship of data: Whether it is necessary to reveal complex relationships between different variables, or simple univariate analysis. The complexity of the decision: If the decision requires the integration of multiple aspects of information, then a deeper insight is required.

[0048] If, based on complexity, depth of insight, and data analysis goals, you determine that your current data analysis involves basic statistical methods and a single variable or a small number of variables, you can use a simple model for data analysis. For example, calculate the average sales of each product, or generate a histogram of sales.

[0049] If, based on complexity, depth of insight, and data analysis goals, you determine that your current data analysis involves correlation analysis of multiple variables, as well as statistical tests and models, you can use the intermediate mode for data analysis, or use the simple mode and the intermediate mode for data analysis. For example, use linear regression to analyze the impact of advertising spending on sales, or perform correlation analysis to identify relationships between variables.

[0050] If, based on complexity, depth of insight, and data analysis goals, it is determined that the current data analysis involves complex models and algorithms, such as machine learning, predictive analysis, etc., then the data analysis can be performed in advanced mode, or in simple mode, intermediate mode, and advanced mode, or in intermediate mode and advanced mode. For example, use random forests or neural networks for sales forecasting, or implement cluster analysis to identify customer segments.

[0051] The embodiment of the present invention selects the data analysis mode according to the complexity of the data, the data analysis goal and the required depth of insight, gradually from simple to complex, which can help users use data more effectively and gradually deepen the analysis.

[0052] Based on the above embodiment, the step of acquiring target data related to a specific field includes: Step 110, acquiring numerical data and text data related to the specific field from at least one data source; the text data is used to describe and annotate the numerical data; Step 111, preprocessing the numerical data and the text data to obtain target data related to the specific field.

[0053] The system obtains target data related to a specific field through the data input module. The data input module is responsible for aggregating data from various sources such as databases, application programming interfaces (APIs), spreadsheets, documents, and real-time data streams. The data input module is used to process numerical data and text data to ensure efficient and simple data processing.

[0054] During the data preprocessing stage, the system will perform necessary cleaning on the numerical data, such as processing missing values, removing outliers, etc., so that the data is consistent and comparable in subsequent analysis. For text data, the system will perform basic text processing operations, such as basic keyword extraction, so that text information and numerical data can be quickly associated in subsequent analysis. In addition, the system also has automated data integration capabilities, which can extract relevant data from multiple data sources and integrate them into a unified analysis data set based on the data collection requirements set by the user. In this way, the system can ensure that the input data has sufficient breadth and depth to support complex pattern recognition and insight generation processes.

[0055] Ultimately, the preprocessed numerical data and text data will be passed to the subsequent analysis module to provide high-quality input data for the pattern recognition module, thereby improving the efficiency and accuracy of model training and recognition.

[0056] The embodiment of the present invention can automatically aggregate data from multiple sources through the data input module, saving manual collection time and improving data processing speed; by cleaning and standardizing numerical data, it can ensure the consistency of data during analysis and reduce errors; process text data to provide background information for numerical data to help better understand the meaning of the data; it can automatically extract and integrate data according to user needs to enhance flexibility and adaptability; ensure that the input data has breadth and depth, thereby improving the effect of pattern recognition and insight generation, and ultimately improving the quality of decision-making.

[0057] Based on the above embodiment, the data analysis results are interpreted and sorted to obtain insight data of the data analysis results, and after the insight data is output, the following steps are included: Step 310, obtaining user feedback data on the insight data; Step 311, adjust the data analysis model or model parameters according to the feedback data; the data analysis model includes a machine learning model and a deep learning model.

[0058] The system processes user feedback on insight data through the learning and feedback module. The learning and feedback module is a key part of the continuous optimization and evolution of the system, ensuring that the system can continuously adapt to user needs and improve the accuracy and relevance of analysis results. By integrating the user feedback mechanism, this module realizes a dynamic closed-loop learning process, allowing the system to not only provide the best solution at the moment, but also become smarter and more efficient over time.

[0059] When users use the insights or predictions generated by the system, they can evaluate the relevance, accuracy, and actual application effects of these outputs. User feedback can take many forms, including ratings, text comments, marking errors, or making suggestions for improvement. After the system collects this feedback, it will conduct an in-depth analysis to identify models or algorithms that perform well in specific situations and what needs to be further optimized. Based on the analysis of user feedback, the system will automatically adjust its internal machine learning models and algorithm parameters. This adjustment may include retraining the model, selecting different features, or adjusting the model's hyperparameters. For models that perform poorly, the system will reduce their weight in future predictions or replace them with new models that are more suitable for current data and goals. In addition, the system can also identify user preferences and needs through feedback data, so as to better customize the output results during the next analysis.

[0060] The learning and feedback module also supports multiple rounds of iterative optimization. As more user feedback is accumulated, the system will continuously update and expand its knowledge base and model library, making it more accurate and efficient when dealing with similar problems. Through this continuous self-learning and adjustment, the system gradually improves its understanding of specific areas or specific user needs, thereby providing insights and suggestions that are increasingly in line with user expectations.

[0061] In addition, the learning and feedback module is highly transparent and explainable. Users can view the system's improvement measures and adjustment strategies after receiving feedback, which not only increases users' trust in the system, but also helps users better understand the system's working principles and improvement process. Through this continuous learning and feedback cycle, the system can remain efficient in a changing environment and always provide users with the best solutions.

[0062] Through user feedback, the embodiment of the present invention can enable the system to continuously adjust and improve the model to enhance accuracy and relevance; identify user preferences, provide more customized insights and suggestions, and meet specific needs; the system can flexibly respond to changing user needs and data environments over time; users can view improvement measures to enhance their trust and understanding of the system; through multiple rounds of iterations, more accurate analysis results are provided to help users make more informed decisions.

[0063] In order to further explain the data analysis method provided by the present invention, refer to the following examples.

[0064] The embodiments of the present invention specifically provide an intelligent system and method for discovering knowledge from chaotic data using a large model. The method uses a large model to discover hidden relationships, correlations and insights from a large amount of chaotic data, aiming to help researchers, financial analysts and other professionals discover valuable knowledge that can be applied in practical scenarios, including scientific discoveries, financial forecasts and other laws that can bring economic benefits.

[0065] refer to Figure 2 , by designing six modules and using large models to assist artificial intelligence algorithms to discover hidden relationships, correlations and insights from large amounts of chaotic data, thereby achieving user-set goals. The specific steps include: Step 1: Data input module, which is used to collect user data and perform data cleaning; Step 2: Goal setting module, which further processes the input data according to the goals set by the user; Step 3: Pattern recognition module, which uses advanced artificial intelligence algorithms to explore and identify hidden patterns in data through hierarchical analysis methods (simple, intermediate, and advanced); hidden patterns refer to regularities and relationships in the data that are not easily observed or identified directly. These patterns may involve complex variable interactions, trends, anomalies, or potential causal relationships. Hidden patterns can reveal important information behind the data and help users make more accurate decisions.

[0066] Step 4: Insight generation module, which generates the identified relevant knowledge in the form of insights, sorts them according to the degree of achieving the goal, and presents them to the user; Step 5: Visualization module, which enables users to interact with the system through the interface; Step 6: Feedback and learning module, which is used to provide feedback from users. The opinions are used to fine-tune the model to adaptively improve prediction accuracy and relevance.

[0067] The embodiments of the present invention utilize a large model to discover hidden relationships, correlations, and insights from a large amount of chaotic data, and present these data to users in an actionable and easy-to-understand manner, making it easier for users to conduct research.

[0068] The data analysis device provided by the present invention is described below. The data analysis device described below and the data analysis method described above can be referenced to each other.

[0069] refer to Figure 3 The data analysis device provided by the present invention includes a core data extraction module 301, a data analysis module 302 and an insight data output module 303.

[0070] The core data extraction module 301 is used to obtain target data related to a specific field and extract core data associated with the data analysis target set by the user from the target data; The data analysis module 302 is used to perform hierarchical progressive analysis on the core data to obtain data analysis results; The insight data output module 303 is used to interpret and organize the data analysis results, obtain insight data of the data analysis results, and output the insight data.

[0071] The data analysis device provided by the embodiment of the present invention obtains target data related to a specific field, extracts core data associated with the data analysis target set by the user from the target data; performs hierarchical progressive analysis on the core data to obtain data analysis results; interprets and organizes the data analysis results to obtain insight data of the data analysis results, and outputs the insight data. The present invention extracts useful data from large-scale and chaotic data through hierarchical progressive analysis and insight analysis, thereby obtaining the relationships, correlations and insights hidden in the chaotic data, and achieving the data analysis target set by the user.

[0072] In one embodiment, the data analysis module 302 is specifically configured to: Using a simple mode, basic data analysis is performed on the core data to obtain first analysis data; If the first analysis data does not match the data required by the data analysis target, at least one machine learning model in the intermediate mode is used to perform data analysis on the core data to obtain second analysis data; If the second analysis data does not match the data required by the data analysis target, at least one deep learning model in the advanced mode is used to perform data analysis on the core data to obtain the data analysis result.

[0073] In one embodiment, the data analysis module 302 is further configured to: Determine the complexity of the target data and the required depth of insight; the depth of insight represents the degree of multi-level interpretation of the data; Determining at least one analysis mode according to the complexity, the insight depth, and the data analysis goal; According to at least one of the analysis modes, data analysis is performed on the core data to obtain the data analysis result.

[0074] In one embodiment, the specific fields include at least scientific research, finance, healthcare and social media. The core data extraction module 301 is further used to: Acquire numerical data and text data related to the specific field from at least one data source; the text data is used to describe and annotate the numerical data; The numerical data and the text data are preprocessed to obtain target data related to the specific field.

[0075] In one embodiment, the insight data output module 303 is specifically configured to: Conducting an in-depth interpretation of the data analysis results to identify key data in the data analysis results; The key data is converted into the insight data.

[0076] In one embodiment, the insight data output module 303 is further configured to: Obtaining user feedback data on the insight data; According to the feedback data, the data analysis model or model parameters are adjusted; the data analysis model includes a machine learning model and a deep learning model.

[0077] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4 As shown, the electronic device may include: a processor 410, a communications interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communications interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the data analysis method, which includes: obtaining target data related to a specific field, extracting core data associated with the data analysis target set by the user from the target data; performing hierarchical progressive analysis on the core data to obtain data analysis results; interpreting and arranging the data analysis results to obtain insight data of the data analysis results, and outputting the insight data.

[0078] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0079] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the data analysis method provided by the above-mentioned methods, which includes: obtaining target data related to a specific field, extracting core data associated with the data analysis target set by the user from the target data; performing hierarchical progressive analysis on the core data to obtain data analysis results; interpreting and organizing the data analysis results to obtain insight data of the data analysis results, and outputting the insight data.

[0080] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it is implemented to execute the data analysis method provided by the above-mentioned methods, the method comprising: obtaining target data related to a specific field, extracting core data associated with the data analysis target set by the user from the target data; performing hierarchical progressive analysis on the core data to obtain data analysis results; interpreting and organizing the data analysis results to obtain insight data of the data analysis results, and outputting the insight data.

[0081] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0082] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0083] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data analysis method, characterized in that: include: Acquire target data related to a specific field, and extract core data associated with the data analysis target set by the user from the target data; Performing hierarchical progressive analysis on the core data to obtain data analysis results; The data analysis results are interpreted and sorted to obtain insight data of the data analysis results, and the insight data is output.

2. The data analysis method according to claim 1, characterized in that: The step of performing hierarchical progressive analysis on the core data to obtain data analysis results includes: Using a simple mode, basic data analysis is performed on the core data to obtain first analysis data; If the first analysis data does not match the data required by the data analysis target, at least one machine learning model in the intermediate mode is used to perform data analysis on the core data to obtain second analysis data; If the second analysis data does not match the data required by the data analysis target, at least one deep learning model in the advanced mode is used to perform data analysis on the core data to obtain the data analysis result.

3. The data analysis method according to claim 1, characterized in that: After acquiring target data related to a specific field and extracting core data associated with a data analysis target set by a user from the target data, the method further includes: Determine the complexity of the target data and the required depth of insight; the depth of insight represents the degree of multi-level interpretation of the data; Determining at least one analysis mode according to the complexity, the insight depth, and the data analysis goal; According to at least one of the analysis modes, data analysis is performed on the core data to obtain the data analysis result.

4. The data analysis method according to claim 1, characterized in that: The specific fields include at least scientific research, finance, health care and social media; The obtaining of target data related to a specific field includes: Acquire numerical data and text data related to the specific field from at least one data source; the text data is used to describe and annotate the numerical data; The numerical data and the text data are preprocessed to obtain target data related to the specific field.

5. The data analysis method according to claim 1, characterized in that: The interpreting and collating of the data analysis results to obtain insight data of the data analysis results includes: Conducting an in-depth interpretation of the data analysis results to identify key data in the data analysis results; The key data is converted into the insight data.

6. The data analysis method according to claim 1, characterized in that: The data analysis results are interpreted and sorted to obtain insight data of the data analysis results, and after the insight data is output, the method includes: Obtaining user feedback data on the insight data; According to the feedback data, the data analysis model or model parameters are adjusted; the data analysis model includes a machine learning model and a deep learning model.

7. A data analysis device, characterized in that: include: A core data extraction module is used to obtain target data related to a specific field and extract core data associated with the data analysis target set by the user from the target data; A data analysis module is used to perform hierarchical progressive analysis on the core data to obtain data analysis results; The insight data output module is used to interpret and organize the data analysis results, obtain insight data of the data analysis results, and output the insight data.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the data analysis method according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the data analysis method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the data analysis method according to any one of claims 1 to 6 is implemented.

Images