Intelligent data analysis method and system based on AI and storage medium

Abstract

The invention discloses an intelligent data analysis method and system based on AI and a storage medium, and belongs to the technical field of data analysis, and the method comprises the steps: obtaining monitoring equipment information of a data source, and obtaining a data label corresponding to the data source based on an equipment type; judging whether the analysis instruction contains an equipment label or not, if yes, obtaining monitoring data corresponding to the equipment label in the analysis instruction, and if not, obtaining an analysis purpose of the analysis instruction, and extracting standard equipment corresponding to the analysis purpose to obtain to-be-analyzed data; abnormal values of the to-be-analyzed data are checked and screened to generate standard analysis data, a model resource library is constructed based on an AI technology, and an optimal analysis model is extracted from the model resource library based on data features and analysis instructions; and inputting the standard analysis data into the optimal analysis model for training and learning, outputting an analysis result, and completing an intelligent decision corresponding to the analysis lexical command. According to the invention, the accuracy of data analysis and the diversity of data analysis types can be improved.

Description

Technical Field

[0001] The present invention belongs to the field of data analysis technology, and specifically relates to an AI-based intelligent data analysis method, system and storage medium. Background Art

[0002] Artificial Intelligence (AI) can provide advanced algorithms and models to process, analyze and interpret large amounts of data. Therefore, the application of AI in the field of data analysis has become more extensive and in-depth, providing more powerful data-driven solutions for all walks of life.

[0003] Similar prior art includes a Chinese patent application with publication number CN117785920A, which discloses an intelligent data analysis and processing method, device, equipment and medium. The method includes calling a client API interface through a shared platform; the client API interface obtains input information submitted by the user, and combines with the template engine to integrate the input information and the corresponding prompt word template and input it to the AI ​​engine; the AI ​​engine generates SQL statements based on the input information and the corresponding prompt word template, and uses the data processing engine to execute the SQL statements to perform data query operations and generate data query results. Advantages of this invention: The entire data analysis and processing process is automatically completed by the client API interface, template engine, AI engine and data processing engine, which can effectively lower the user's usage threshold and improve data analysis efficiency, so that users without professional skills can also perform efficient data analysis and processing, and can improve the accuracy of data analysis. There is also a Chinese patent application with publication number CN117216419A, which discloses a data analysis method based on AI technology. The method includes the following steps: obtaining user data; using artificial intelligence technology to perform dynamic behavior analysis on user data to generate user dynamic behavior data; performing behavior pattern analysis on user dynamic behavior data to generate user behavior pattern data; performing time series analysis on user dynamic behavior data according to user behavior pattern data to generate user behavior time series data; performing emotion fluctuation recognition on user dynamic behavior data based on user behavior time series data to obtain user emotion fluctuation data; using computer vision technology to perform user portrait fitting on user dynamic behavior data according to user emotion fluctuation data to construct user dynamic portrait; performing eye trajectory analysis on user dynamic portrait to generate user personality preference data. The invention realizes efficient and accurate user data analysis.

[0004] However, the above-mentioned existing technologies only analyze a single type of data, which limits the diversity of artificial intelligence technology. In actual situations, it is necessary to collect necessary analysis data based on analysis commands and perform combined analysis on different types of analysis data. By generating the corresponding optimal analysis model through AI technology, the accuracy of data analysis and the accuracy of intelligent decision-making can be improved, and the error of data analysis can be reduced by extracting necessary analysis data. Summary of the invention

[0005] To solve the above problems, the present invention provides an AI-based intelligent data analysis method, system and storage medium to solve the problems in the prior art.

[0006] In order to achieve the above-mentioned object of the invention, the present invention proposes an AI-based intelligent data analysis method, comprising:

[0007] Acquire monitoring device information of the data source, the monitoring device information including device label, device type and device location, and acquire a data label corresponding to the data source based on the device type;

[0008] Setting an acquisition and transmission mode, receiving and identifying an analysis command, determining whether the analysis command contains the device tag, if so, obtaining monitoring data corresponding to the device tag in the analysis command based on the acquisition and transmission mode, and setting the data as the data to be analyzed; otherwise, obtaining an analysis purpose of the analysis command, comparing the analysis purpose with the monitoring device information, extracting standard devices corresponding to the analysis purpose from all monitoring devices, obtaining monitoring data corresponding to the standard devices based on the acquisition and transmission mode, and setting the data as the data to be analyzed;

[0009] Check the outliers of the data to be analyzed based on the data labels, filter the data to be analyzed based on the outliers to generate standard analysis data, build a model resource library based on AI technology, extract data features of the standard analysis data, and extract the best analysis model from the model resource library based on the data features and the analysis command;

[0010] The standard analysis data is input into the optimal analysis model for training and learning, and an analysis result is output based on the analysis command, and an intelligent decision corresponding to the analysis command is completed based on the analysis result.

[0011] Further, the acquisition transmission mode is set based on the following steps:

[0012] The device corresponding to the data source is set as a monitoring device, a preset number of intermediate devices are built for the monitoring device, the monitoring data corresponding to the monitoring device is stored in any intermediate device, the monitoring device and all the corresponding intermediate devices are combined to construct a status analysis table, the status analysis table records and updates the storage occupancy corresponding to all the intermediate devices, the status analysis table switches the storage state of any of the intermediate devices based on the storage occupancy, and the collection and storage process of the monitoring device is completed;

[0013] After identifying the analysis command, a data request is sent to the intermediate device based on the device tag, and the monitoring data stored in all the intermediate devices are extracted in sequence based on the data request to complete the transmission process of the monitoring data. The collection and storage process and the transmission process are combined and set to the collection and transmission mode.

[0014] Furthermore, generating the data to be analyzed includes the following steps:

[0015] The device tag or the standard device in the analysis command is set as the target device, and the data request is sent to the target device, wherein the data request includes the data tag and time information, and all the intermediate devices are checked based on the data request. If the intermediate device meets the data request, the corresponding stored monitoring data is output, and the monitoring data corresponding to all the intermediate devices are combined based on the time information and set as the data to be analyzed.

[0016] Further, judging whether the analysis command includes the device tag is based on the following steps:

[0017] Analyze and interpret the analysis command based on a semantic analysis algorithm, generate an extended command, set an interpretation tag, the interpretation tag includes a data analysis object and a data analysis purpose, and extract the analysis object command corresponding to the data analysis object and the analysis purpose command corresponding to the data analysis purpose from the extended command based on the interpretation tag;

[0018] Based on the character similarity, it is determined whether the character corresponding to the device tag exists in the analysis object command. If so, it is determined that the analysis command contains the device tag. Otherwise, the extended command is set as the analysis purpose corresponding to the analysis command.

[0019] Furthermore, extracting standard equipment corresponding to the analysis purpose from all monitoring equipment includes the following steps:

[0020] Match the characters corresponding to the analysis purpose with the characters corresponding to any monitoring device information respectively, calculate the device position of any monitoring device and the word vector similarity between the device type and the analysis purpose based on the semantic analysis algorithm, and set them as first values. If both first values ​​are greater than a first threshold, set any monitoring device as the standard device.

[0021] Furthermore, extracting the best analysis model from the model resource library based on the data features and the analysis command comprises the following steps:

[0022] Setting a model tag, the AI ​​technology generates a plurality of data analysis models based on the model tag, and all the data analysis models are arranged and combined based on the model tag to generate the model resource library, wherein the model tag includes a model name, a model function, a model defect, and a model analysis process;

[0023] Based on the data preprocessing process, detect outliers between the data to be analyzed corresponding to the same data label, delete the outliers, and set the remaining data as the standard analysis data;

[0024] Setting a feature analysis algorithm corresponding to the data tag based on the data preprocessing algorithm, and extracting the data feature of the standard analysis data based on the feature analysis algorithm;

[0025] In the model analysis process, a model that meets the data features and the data labels is extracted and set as an alternative model. Based on the semantic analysis algorithm, the word vector similarity between the analysis command and the model function of any alternative model is obtained and set as a second value. The model with the largest second value is set as the optimal analysis model.

[0026] Furthermore, the intelligent decision-making corresponding to the analysis command based on the analysis result includes the following steps:

[0027] The optimal analysis model is trained based on the standard analysis data to generate an estimation model, the estimation model outputs an estimation sequence result based on the data features, a monitoring mode of the standard analysis data is obtained based on the estimation sequence result, and is set as the analysis result;

[0028] A monitoring decision table is constructed, and corresponding decision information is obtained from the monitoring decision table based on the analysis result and the estimation sequence result, and is set as the intelligent decision.

[0029] The present invention also provides an AI-based intelligent data analysis system, which is used to implement the above-mentioned AI-based intelligent data analysis method, and the system mainly includes:

[0030] The device processing module obtains the monitoring device information of the data source, wherein the monitoring device information includes the device tag, device type and device location, and obtains the data tag corresponding to the data source based on the device type;

[0031] a data acquisition module, setting an acquisition transmission mode, receiving and identifying an analysis command, determining whether the analysis command contains the device tag, and if so, acquiring the monitoring data corresponding to the device tag in the analysis command based on the acquisition transmission mode, and setting it as the data to be analyzed; otherwise, acquiring the analysis purpose of the analysis command, comparing the analysis purpose with the monitoring device information, extracting standard devices corresponding to the analysis purpose from all monitoring devices, acquiring the monitoring data corresponding to the standard devices based on the acquisition transmission mode, and setting it as the data to be analyzed;

[0032] A model generation module, which checks the outliers of the data to be analyzed based on the data labels, screens the data to be analyzed based on the outliers to generate standard analysis data, builds a model resource library based on AI technology, extracts data features of the standard analysis data, and extracts the best analysis model from the model resource library based on the data features and the analysis command;

[0033] The data analysis module inputs the standard analysis data into the optimal analysis model for training and learning, outputs analysis results based on the analysis command, and completes the intelligent decision corresponding to the analysis command based on the analysis results.

[0034] The present invention also provides a computer storage medium, which stores program instructions, wherein when the program instructions are executed, the device where the computer storage medium is located is controlled to execute the above-mentioned AI-based intelligent data analysis method.

[0035] Compared with the prior art, the beneficial effects of the present invention are at least as follows:

[0036] The present invention can first determine the way to obtain the data source by marking the monitoring equipment information corresponding to the data source, and then limit the required equipment by identifying the analysis command, and obtain the required data to be analyzed by collecting the transmission mode, which can reduce the problem of data storage capacity overload, and also can accurately obtain the data to be analyzed contained in the analysis command, thereby improving the accuracy of obtaining the data to be analyzed. Finally, a model resource library is constructed through AI technology, and the best analysis model is extracted by data features and data labels, which can improve the accuracy of data analysis and the diversity of data analysis types.

[0037] The present invention also realizes intelligent decision-making based on analysis results in data analysis by constructing a monitoring decision table, thereby improving the intelligence of data analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is a flowchart of the steps of an AI-based intelligent data analysis method of the present invention;

[0039] Figure 2 The figure is a schematic diagram of the structure of an AI-based intelligent data analysis system of the present invention. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0041] It is understood that the terms "first", "second", etc. used in this application may be used herein to describe various elements, but unless otherwise specified, these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, without departing from the scope of this application, a first xx script may be referred to as a second xx script, and similarly, a second xx script may be referred to as a first xx script.

[0042] like Figure 1 As shown, an AI-based intelligent data analysis method includes:

[0043] S1: Obtain monitoring device information of the data source, the monitoring device information includes device label, device type and device location, and obtain the data label corresponding to the data source based on the device type.

[0044] Specifically, in this embodiment, the data source refers to the device destination for receiving data, such as various types of sensor devices, mobile terminals for recording logs, etc. Monitoring device information refers to detailed information of the device corresponding to the data source, wherein the device tag refers to the device ID, the device type refers to the name of the device type, and the device location refers to the device installation information, such as the temperature sensor M1 installed at the north gate of the park, wherein the device tag is M1, the device type is the temperature sensor, and the device location is the north gate of the park. The data tag refers to the data type information of the data collected using the device corresponding to the device type, including but not limited to the data sampling rate and data type, etc., wherein the data type includes but is not limited to images, text, numbers, signals, etc.

[0045] S2: Set the collection and transmission mode, receive and identify the analysis command, and determine whether the analysis command contains a device tag. If so, obtain the monitoring data corresponding to the device tag in the analysis command based on the collection and transmission mode, and set it as the data to be analyzed. Otherwise, obtain the analysis purpose of the analysis command, compare the analysis purpose with the monitoring device information, extract the standard equipment corresponding to the analysis purpose from all monitoring devices, obtain the monitoring data corresponding to the standard equipment based on the collection and transmission mode, and set it as the data to be analyzed.

[0046] Specifically, in this embodiment, the acquisition and transmission mode refers to the acquisition, storage and transmission modes corresponding to the various devices corresponding to the data source when collecting data, and the analysis command refers to the command used to describe the data analysis requirements, including but not limited to character commands, voice commands and database commands, etc., which can be received and recognized by the processing platform. If the analysis command contains a device tag, it means that the analysis command has set the data to be analyzed, and the acquisition data of the device corresponding to the device tag can be directly retrieved from the acquisition and transmission mode, that is, the monitoring data. For example, the analysis command L1 is: analyze the acquisition data of temperature sensors M1 and M2, then the monitoring data corresponding to the device tags M1 and M2 are retrieved based on the acquisition and transmission mode. If the analysis command does not contain a device tag, it is necessary to perform semantic analysis on the analysis command and extract the analysis purpose, wherein the analysis purpose refers to the analysis requirements, and the standard equipment can be extracted by comparing the monitoring equipment information, wherein the standard equipment refers to the equipment list involved in the analysis command, for example, the analysis command L2 is: "Monitor the temperature distribution of the park", then based on the semantic analysis algorithm, "monitoring", "park" and "temperature" are expanded and combined into the analysis purpose: "monitor and analyze the temperature of the park". Standard equipment can be extracted from the information corresponding to the equipment type and equipment location in the monitoring equipment information. For example, all "temperature" type equipment belonging to "park" in the equipment location can be set as standard equipment. The data to be analyzed is a data set collected by the monitoring data of each device through the collection transmission mode.

[0047] S3: Check the outliers of the data to be analyzed based on the data labels, filter the data to be analyzed based on the outliers to generate standard analysis data, build a model resource library based on AI technology, extract the data features of the standard analysis data, and extract the best analysis model in the model resource library based on the data features and analysis commands.

[0048] Specifically, an outlier refers to a data point in the data to be analyzed that deviates significantly from other observed values. After filtering out the outliers, standard analysis data can be generated, where standard analysis data is complete data used for data analysis by analysis commands, and can include data sources corresponding to multiple data labels. AI technology refers to artificial intelligence technology, including machine learning and deep learning, which can automatically create, optimize and select multiple models to solve complex data analysis problems. Model resource library refers to a resource library for building multiple models for data analysis. Data features refer to attributes or variables used to describe or characterize data points in standard analysis data. They are the basic elements that constitute standard analysis data and are used to capture and describe the essential characteristics of data from different angles. In data analysis, machine learning and statistical modeling, data features are the basis for understanding and analyzing data. Therefore, the best analysis model can be extracted from the model resource library by combining data features with analysis commands, where the best analysis model is the model used to perform data analysis corresponding to the analysis command.

[0049] S4: Input the standard analysis data into the optimal analysis model for training and learning, output the analysis results based on the analysis commands, and complete the intelligent decision corresponding to the analysis commands based on the analysis results.

[0050] Specifically, in this embodiment, the optimal analysis model is generated by combining neural network models corresponding to various AI technologies. Therefore, it is necessary to input standard analysis data into the optimal analysis model for training and learning. The training and learning process involves adjusting model parameters, selecting the best model hyperparameters, and outputting the analysis results corresponding to the analysis command. Intelligent decision-making refers to a reference solution for solving data analysis needs in analysis commands, which can realize automated decision-making and improve the efficiency, accuracy and reliability of decision-making.

[0051] As a preferred technical solution of the present invention, the acquisition transmission mode is set based on the following steps:

[0052] The device corresponding to the data source is set as the monitoring device, a preset number of intermediate devices are built for the monitoring device, the monitoring data corresponding to the monitoring device is stored in any intermediate device, the monitoring device and all corresponding intermediate devices are combined to construct a status analysis table, the status analysis table records and updates the storage occupancy corresponding to all intermediate devices, the status analysis table switches the storage status of any intermediate device based on the storage occupancy, and the collection and storage process of the monitoring device is completed.

[0053] After identifying and analyzing the command, a data request is sent to the intermediate device based on the device tag. Based on the data request, the monitoring data stored in all the intermediate devices are extracted in turn to complete the transmission process of the monitoring data. The collection and storage process and the transmission process are combined and set to the collection and transmission mode.

[0054] Specifically, the monitoring device corresponds to the monitoring device information one by one. The intermediate device refers to the edge storage device of the monitoring device, which is used to receive and store the monitoring data of the monitoring device, wherein the preset number is an integer greater than or equal to 2. The capacity memory when the data is stored in the intermediate device can be reduced. The state analysis table is a table for recording the usage status of all monitoring devices and the corresponding intermediate devices. The storage occupancy rate refers to the capacity ratio of the intermediate device. The larger the storage occupancy rate, the larger the capacity ratio of the intermediate device, and the lower the corresponding storage efficiency. The state analysis table can switch the storage status of different intermediate devices corresponding to the same monitoring device based on the comparison between the storage occupancy rate and the preset threshold. For example, the same monitoring device has two intermediate devices P1 and P2. If the storage occupancy rate of the intermediate device P1 is greater than the preset threshold, the intermediate device P1 is turned off, and the storage status of the intermediate device P2 is started, so that the monitoring data of the monitoring device is stored in the intermediate device P2. In this way, the collection and storage process of the monitoring device can be completed.

[0055] After identifying and analyzing the command, a data request can be sent to the corresponding monitoring device. After the monitoring device receives the data request, the corresponding intermediate device can transmit and extract the stored monitoring data. In this way, the transmission process of the monitoring data can be completed. The collection and storage process and the transmission process can be combined to set the collection and transmission mode. The collection and transmission mode can improve the stability of the data collection and transmission process and reduce the loss of data storage.

[0056] As a preferred technical solution of the present invention, generating the data to be analyzed includes the following steps:

[0057] The device tag or standard device in the analysis command is set as the target device, and a data request is sent to the target device, wherein the data request includes a data tag and time information. All intermediate devices are checked based on the data request. If the intermediate device meets the data request, the corresponding stored monitoring data is output. The monitoring data corresponding to all intermediate devices are combined based on the time information and set as the data to be analyzed.

[0058] Specifically, in this embodiment, the target device refers to the device selected and obtained by the analysis command from the devices corresponding to all data sources, including the device tag corresponding to the analysis command or the device corresponding to the standard device. After identifying the analysis command, a data request is sent to the target device, wherein the time information refers to the time range corresponding to the data analysis required in the data request. All intermediate devices corresponding to the target device are proofread by the data request, wherein the proofreading refers to whether the data type and time information are consistent with the data request. The monitoring data corresponding to all intermediate devices that meet the data request are obtained, and after being combined in sequence by time information, the data to be analyzed can be generated.

[0059] As a preferred technical solution of the present invention, it is determined whether the analysis command contains a device tag based on the following steps:

[0060] The analysis command is analyzed and interpreted based on the semantic analysis algorithm, an extended command is generated, and an interpretation label is set. The interpretation label includes a data analysis object and a data analysis purpose. Based on the interpretation label, the analysis object command corresponding to the data analysis object and the analysis purpose command corresponding to the data analysis purpose are respectively extracted from the extended command.

[0061] Based on the character similarity, it is determined whether there is a character corresponding to the device label in the analysis object command. If so, it is determined that the analysis command contains the device label. Otherwise, the extended command is set as the analysis purpose corresponding to the analysis command.

[0062] Specifically, in this embodiment, the semantic analysis algorithm refers to a series of computational methods and techniques for understanding and interpreting the meaning, intention and emotion in natural language, so as to achieve a more complete natural language processing (NLP) task. The algorithm has functions including but not limited to semantic role labeling, text summarization, semantic expansion and natural language reasoning. The extended command refers to the command generated by using the semantic analysis algorithm to perform natural language reasoning on the analysis command. For example, if the text characters of the analysis command are: "Temperature monitoring of the park", the text characters corresponding to the extended command are: "Predict and analyze the temperature of park A and park B respectively". The interpretation label refers to the label corresponding to the text information that provides a basis for data analysis, wherein the data analysis object refers to the text information corresponding to the target object, and the data analysis purpose refers to the text information corresponding to the specific goal or problem that the data analysis hopes to achieve or solve. The analysis object command refers to the command information belonging to the data analysis object in the extended command, and the analysis purpose command refers to the command information belonging to the data analysis purpose in the extended command. For example, in the extended command "perform forecast analysis on the temperature of Park A and Park B respectively", the analysis object commands are "temperature of Park A" and "temperature of Park B" respectively, and the analysis purpose command is "forecast analysis".

[0063] Character similarity refers to an indicator that measures the similarity between two character strings (text sequences) in character composition, order or structure. In this embodiment, the character similarity Q is calculated using the first formula, which is: Among them, q 2 is the number of characters corresponding to any device label, q 1It is the number of identical characters between the analysis object command and the device label. If the character similarity is 1, it means that the analysis object command contains the same characters as the device label, and it can be determined that the analysis object command contains the device label, and all device labels are compared for character similarity. Otherwise, it is determined that it does not exist. If it exists, it means that the monitoring device that needs to be analyzed is set in the analysis command, and the corresponding monitoring device can be directly selected according to the existing device label. If it does not exist, the extended command can be directly set to the character corresponding to the analysis purpose.

[0064] As a preferred technical solution of the present invention, extracting standard equipment corresponding to the analysis purpose from all monitoring equipment includes the following steps:

[0065] Match the characters corresponding to the analysis purpose with the characters corresponding to any monitoring device information respectively, calculate the device position of any monitoring device and the word vector similarity between the device type and the analysis purpose based on the semantic analysis algorithm, and set them as the first value. If both first values ​​are greater than the first threshold, set any monitoring device as a standard device.

[0066] Specifically, in this embodiment, if there is no character corresponding to the device label directly in the analysis command, it is necessary to extract the standard equipment through the analysis purpose corresponding to the analysis command. The character corresponding to the analysis purpose is matched with the character contained in any monitoring device information, and the word vector between the two can be extracted through the Word2Vec model in the semantic analysis algorithm, and the corresponding word vector similarity is calculated by cosine similarity, that is, a first value is generated. If both first values ​​are greater than the first threshold, it means that the device location and device type match the analysis purpose. For example, the analysis purpose is: "Predict and analyze the temperature of Park A and Park B respectively". Through the word vector similarity, it can be extracted that the "temperature of Park A" has devices T1 and T2, and the "temperature of Park B" has device T3. Then, devices T1 and T2 are set as the first group of standard devices, and device T3 is set as the second group of standard devices. The monitoring data corresponding to the two groups of standard devices are obtained respectively, and predictive analysis is performed.

[0067] As a preferred technical solution of the present invention, extracting the best analysis model from the model resource library based on data features and analysis commands includes the following steps:

[0068] Set model labels. AI technology generates multiple data analysis models based on model labels. All data analysis models are arranged and combined based on model labels to generate a model resource library. Model labels include model name, model function, model defects and model analysis process.

[0069] Based on the data preprocessing process, outliers between the data to be analyzed corresponding to the same data labels are checked, and the outliers are deleted, and the remaining data are set as standard analysis data.

[0070] A feature analysis algorithm corresponding to the data label is set based on the data preprocessing algorithm, and data features of the standard analysis data are extracted based on the feature analysis algorithm.

[0071] In the model analysis process, a model that meets the data characteristics and data labels is extracted and set as an alternative model. Based on the semantic analysis algorithm, the word vector similarity between the analysis command and the model function of any alternative model is obtained and set as the second value. The model with the largest second value is set as the best analysis model.

[0072] Specifically, in this embodiment, the model label refers to the label information used to describe each model, wherein the model name refers to the model type information generated using AI technology, the model function refers to the main technical function implemented by the model during data analysis, the model defect refers to the shortcomings of the model, and the model analysis process refers to the step information when the model performs data analysis. A variety of data analysis models can be generated through AI technology. These models are widely used in various industries and fields to solve complex data analysis problems, including but not limited to prediction models, classification models, clustering models, anomaly detection models, image processing models, deep learning models, and multimodal learning models. Based on the model labels, the information contained in various data analysis models is summarized and arranged to generate a model resource library.

[0073] The data preprocessing process refers to a series of operations performed on the data before data analysis and processing, which can make the data more suitable for statistical analysis, machine learning or other forms of data mining, including but not limited to data cleaning, data integration, data dimensionality reduction and feature engineering. In this step, data preprocessing refers to the data cleaning algorithm, which uses the data cleaning algorithm to remove outliers from the data to be analyzed corresponding to any identical data label, and sets the remaining data as standard analysis data, where outliers refer to erroneous data, missing values ​​and other data that do not conform to data specifications.

[0074] The feature analysis algorithm refers to an algorithm used for feature extraction. Different feature analysis algorithms are selected according to different data labels so that the generated data features are applicable. For example, if the data label contains the image type, the feature analysis algorithm can be set to the SIFT scale-invariant feature transformation algorithm.

[0075] The model that meets the data feature refers to the name of the model that can be used for data analysis of the data feature. It is extracted through the model analysis process in the model tag. If the model analysis process includes analyzing and processing the data feature and the data label, the data analysis model corresponding to the model analysis process is extracted as the candidate model. Since most models can analyze and process the same data feature, and at the same time, a set of standard analysis data may have multiple data features, the number of candidate models may be large. Therefore, they can be filtered through the information corresponding to the model function in the model tag. Similarly, by obtaining the word vector similarity between the analysis purpose in the analysis command and the model function through the semantic analysis algorithm, the model that best meets the analysis command can be selected from the candidate models, that is, the best analysis model. The best analysis model can complete the data analysis corresponding to the analysis command through the data features corresponding to the standard analysis data.

[0076] As a preferred technical solution of the present invention, completing intelligent decision-making corresponding to the analysis command based on the analysis result includes the following steps:

[0077] The best analysis model is trained based on standard analysis data to generate an estimation model. The estimation model outputs estimation sequence results based on data features. The monitoring mode of the standard analysis data is obtained based on the estimation sequence results and set as the analysis result.

[0078] Construct a monitoring decision table, obtain corresponding decision information in the monitoring decision table based on the analysis results and estimated sequence results, and set it as an intelligent decision.

[0079] Specifically, in this embodiment, all standard analysis data are input into the optimal analysis model, and the various hyperparameters of the optimal analysis model can be adjusted and set by the learning and training process to generate an estimation model, wherein the estimation model is a model after the parameters of the optimal analysis model are adjusted. The estimation model intelligently outputs the estimation sequence results based on the data features of the standard analysis data, wherein the estimation sequence results refer to the output of the learning algorithm in the estimation model after analyzing the data features in the standard analysis data. The monitoring mode is to identify the potential relationships and rules in the estimation sequence results through a specific algorithm in the estimation model.

[0080] The estimated sequence results can be used in various scenarios, such as system fault diagnosis, network security monitoring, predictive maintenance of industrial systems, and intelligent monitoring of parks. Therefore, by constructing a monitoring decision table, the corresponding decision information can be quickly located based on the estimated sequence results to generate intelligent decisions. Among them, the monitoring decision table is a plan table set according to different data sources, and the decision information is the corresponding solution in the monitoring decision table. For example, the analysis results can determine the type of data source that needs to be monitored. If the estimated sequence results exceed the maximum threshold corresponding to the data source in the monitoring decision table, the corresponding decision information can be set as: send an early warning message and set it as an intelligent decision. This step can improve the accuracy, intelligence, and applicability of data analysis.

[0081] like Figure 2 As shown, the present invention also provides an AI-based intelligent data analysis system, which is used to implement the above-mentioned AI-based intelligent data analysis method, and the system mainly includes:

[0082] The device processing module obtains the monitoring device information of the data source. The monitoring device information includes the device label, device type and device location. The data label corresponding to the data source is obtained based on the device type.

[0083] The data acquisition module sets the collection and transmission mode, receives and identifies the analysis command, and determines whether the analysis command contains the device tag. If so, the monitoring data corresponding to the device tag in the analysis command is obtained based on the collection and transmission mode, and set as the data to be analyzed. Otherwise, the analysis purpose of the analysis command is obtained, the analysis purpose is compared with the monitoring device information, the standard equipment corresponding to the analysis purpose is extracted from all monitoring devices, the monitoring data corresponding to the standard equipment is obtained based on the collection and transmission mode, and set as the data to be analyzed.

[0084] The model generation module checks the outliers of the data to be analyzed based on data labels, screens the data to be analyzed based on the outliers to generate standard analysis data, builds a model resource library based on AI technology, extracts data features of standard analysis data, and extracts the best analysis model in the model resource library based on data features and analysis commands.

[0085] The data analysis module inputs the standard analysis data into the optimal analysis model for training and learning, outputs the analysis results based on the analysis commands, and completes the intelligent decision corresponding to the analysis commands based on the analysis results.

[0086] The present invention also provides a computer storage medium, which stores program instructions, wherein when the program instructions are executed, the device where the computer storage medium is located is controlled to execute the above-mentioned AI-based intelligent data analysis method.

[0087] It should be understood that, although each step in the flow chart of each embodiment of the present invention is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0088] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The above-mentioned program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0089] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0090] The above embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for those of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

[0091] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An AI-based intelligent data analysis method, It is characterized in that The method comprises the following steps: Acquire monitoring device information of the data source, the monitoring device information including device label, device type and device location, and acquire a data label corresponding to the data source based on the device type; Setting an acquisition and transmission mode, receiving and identifying an analysis command, determining whether the analysis command contains the device tag, if so, obtaining monitoring data corresponding to the device tag in the analysis command based on the acquisition and transmission mode, and setting the data as the data to be analyzed; otherwise, obtaining an analysis purpose of the analysis command, comparing the analysis purpose with the monitoring device information, extracting standard devices corresponding to the analysis purpose from all monitoring devices, obtaining monitoring data corresponding to the standard devices based on the acquisition and transmission mode, and setting the data as the data to be analyzed; Check the outliers of the data to be analyzed based on the data labels, filter the data to be analyzed based on the outliers to generate standard analysis data, build a model resource library based on AI technology, extract data features of the standard analysis data, and extract the best analysis model from the model resource library based on the data features and the analysis command; The standard analysis data is input into the optimal analysis model for training and learning, and an analysis result is output based on the analysis command, and an intelligent decision corresponding to the analysis command is completed based on the analysis result.

2. The method according to claim 1, It is characterized in that The acquisition transmission mode is set based on the following steps: The device corresponding to the data source is set as a monitoring device, a preset number of intermediate devices are built for the monitoring device, the monitoring data corresponding to the monitoring device is stored in any intermediate device, the monitoring device and all the corresponding intermediate devices are combined to construct a status analysis table, the status analysis table records and updates the storage occupancy corresponding to all the intermediate devices, the status analysis table switches the storage state of any of the intermediate devices based on the storage occupancy, and the collection and storage process of the monitoring device is completed; After identifying the analysis command, a data request is sent to the intermediate device based on the device tag, and the monitoring data stored in all the intermediate devices are extracted in sequence based on the data request to complete the transmission process of the monitoring data. The collection and storage process and the transmission process are combined and set to the collection and transmission mode.

3. The method according to claim 2, It is characterized in that Generating the data to be analyzed comprises the following steps: The device tag or the standard device in the analysis command is set as the target device, and the data request is sent to the target device, wherein the data request includes the data tag and time information, and all the intermediate devices are checked based on the data request. If the intermediate device meets the data request, the corresponding stored monitoring data is output, and the monitoring data corresponding to all the intermediate devices are combined based on the time information and set as the data to be analyzed.

4. The method according to claim 1, It is characterized in that Determine whether the analysis command contains the device tag based on the following steps: Analyze and interpret the analysis command based on a semantic analysis algorithm, generate an extended command, set an interpretation tag, the interpretation tag includes a data analysis object and a data analysis purpose, and extract the analysis object command corresponding to the data analysis object and the analysis purpose command corresponding to the data analysis purpose from the extended command based on the interpretation tag; Based on the character similarity, it is determined whether the character corresponding to the device tag exists in the analysis object command. If so, it is determined that the analysis command contains the device tag. Otherwise, the extended command is set as the analysis purpose corresponding to the analysis command.

5. The method according to claim 4, It is characterized in that The step of extracting standard equipment corresponding to the analysis purpose from all monitoring equipment comprises the following steps: The characters corresponding to the analysis purpose are matched with the characters corresponding to any monitoring device information respectively, and the device position of any monitoring device and the word vector similarity between the device type and the analysis purpose are calculated based on the semantic analysis algorithm and set as a first value. If both first values ​​are greater than a first threshold, any monitoring device is set as the standard device.

6. The method according to claim 1, It is characterized in that The extracting the best analysis model from the model resource library based on the data features and the analysis command comprises the following steps: Setting a model tag, the AI ​​technology generates a plurality of data analysis models based on the model tag, and all the data analysis models are arranged and combined based on the model tag to generate the model resource library, wherein the model tag includes a model name, a model function, a model defect, and a model analysis process; Based on the data preprocessing process, detect outliers between the data to be analyzed corresponding to the same data label, delete the outliers, and set the remaining data as the standard analysis data; Setting a feature analysis algorithm corresponding to the data tag based on the data preprocessing algorithm, and extracting the data feature of the standard analysis data based on the feature analysis algorithm; In the model analysis process, a model that meets the data features and the data labels is extracted and set as an alternative model. Based on the semantic analysis algorithm, the word vector similarity between the analysis command and the model function of any alternative model is obtained and set as a second value. The model with the largest second value is set as the optimal analysis model.

7. The method according to claim 6, It is characterized in that The intelligent decision-making corresponding to the analysis command based on the analysis result comprises the following steps: The optimal analysis model is trained based on the standard analysis data to generate an estimation model, the estimation model outputs an estimation sequence result based on the data features, a monitoring mode of the standard analysis data is obtained based on the estimation sequence result, and is set as the analysis result; A monitoring decision table is constructed, and corresponding decision information is obtained from the monitoring decision table based on the analysis result and the estimation sequence result, and is set as the intelligent decision.

8. An AI-based intelligent data analysis system, used to implement the method according to any one of claims 1 to 7, It is characterized in that The system includes the following modules: The device processing module obtains the monitoring device information of the data source, wherein the monitoring device information includes the device tag, device type and device location, and obtains the data tag corresponding to the data source based on the device type; a data acquisition module, setting an acquisition transmission mode, receiving and identifying an analysis command, determining whether the analysis command contains the device tag, and if so, acquiring the monitoring data corresponding to the device tag in the analysis command based on the acquisition transmission mode, and setting it as the data to be analyzed; otherwise, acquiring the analysis purpose of the analysis command, comparing the analysis purpose with the monitoring device information, extracting standard devices corresponding to the analysis purpose from all monitoring devices, acquiring the monitoring data corresponding to the standard devices based on the acquisition transmission mode, and setting it as the data to be analyzed; A model generation module, which checks the outliers of the data to be analyzed based on the data labels, screens the data to be analyzed based on the outliers to generate standard analysis data, builds a model resource library based on AI technology, extracts data features of the standard analysis data, and extracts the best analysis model from the model resource library based on the data features and the analysis command; The data analysis module inputs the standard analysis data into the optimal analysis model for training and learning, outputs analysis results based on the analysis command, and completes the intelligent decision corresponding to the analysis command based on the analysis results.

9. A computer storage medium, It is characterized in that The computer storage medium stores program instructions, wherein when the program instructions are executed, the device where the computer storage medium is located is controlled to execute the method according to any one of claims 1 to 7.

Images