338 results about "Data mining algorithm" patented technology

A distributed storage and parallel mining method for state monitoring data includes the steps: defining function service models of a remote substation state monitoring unit and a state monitoring communication front-end processor by means of Web service description language, and exchanging the state monitoring data of electric power equipment in an electric power wide area network environment by a simple object access protocol; storing large-scale state monitoring data redundancy in a distributed file system, creating an index table for a state monitoring data file, inserting the index table into a large-scale structural data table and querying the state monitoring data according to a query request; and generating basic data and multi-dimensional analytical data by extracting, converting and loading to built a data warehouse, and parallelly executing association rules, classification and clustered data mining algorithm by means of MapReduce task decomposition and result summary. The distributed storage and parallel mining method can be used for effectively realizing distributed data exchange, redundant storage and rapid parallel processing for state monitoring information of the mass electric power equipment in an intelligent power network environment.

A framework is provided that enables data mining algorithms to be plugged into it without any change to algorithm software implementations, while still providing all the standard data mining tasks. It may be implemented by the data source provider. It also then allows for the complete separation of data storage and algorithms. When the user initiates a mining session and picks an algorithm for build task or a model for an apply or test task, the framework may become responsible for preparing a set of “prompts” to the user asking him to provide some expression which is specific to the particular kind of data the user is working with.

The invention discloses a clinical data mining analysis and aided decision-making method based on an Internet integrated medical platform, and relates to the technical field of an Internet medical platform. The clinical data mining analysis and aided decision-making method includes data mining analysis and aided decision-making, wherein data mining analysis includes a multidimensional analysis algorithm module, a data mining algorithm module and a deep learning algorithm module; and aided remote decision-making includes four parts: a prediction module based on index parameters, a prediction module based on inspection report texts, a model training module and a structurized module. The clinical data mining analysis and aided decision-making method based on an Internet integrated medical platform selects several diseases as research objects for data collection and analysis, such as hyperthyroidism, diabetes, thyroid nodules and breast tumors, and collects and integrates clinical medicaldata depended the integrated platform to realize data mining analysis and aided decision-making services for clinical data diseases, such as hyperthyroidism, diabetes, thyroid nodules and breast tumors, so as to provide systematic support for clinical diagnosis of clinicians and disease research by researchers.

The invention discloses a deep leaning-based network intrusion detection and vulnerability scanning method and devices. The method comprises the steps of collecting malicious sample files and buildinga malicious file database; performing training modeling according to behaviors of malicious files in the malicious file database by using a deep learning algorithm, performing real-time monitored model incremental training according to received new malicious sample files, so as to obtain classified models; simulating and running the malicious sample files in the malicious file database in different environments, and detecting an attack characteristic of the malicious sample files by using IDS; and analyzing the malicious file database by using a data mining algorithm, building a vulnerabilityattack manner characteristic library, generating a network attack package, and scanning network vulnerabilities.

The invention provides an intelligent home optimization system based on data mining, which comprises a user authentication module, a journal memory module, a data extraction module, a data mining algoritic module, a sensor monitoring module, a synchronous renewal module, a database module, a scheduler module, and a communication module. An intelligent home control system is internally added with the function of then data mining algoritic and the communication module which can communicate with other homes to cause that after a home system passes through authentication, the home system counts and analyzes a series of actions of family members after the family members go home, analyzes habits and customs of people by the data mining algoritic so as to obtain some regularity, and then transmits the regularity, namely, associative rules to the scheduler module of the intelligent home control system; the scheduler module transmits a control command to each home which is connected with a central control system; therefore, the system can control home without needing operation of the family members in a humanity intelligent way.

Provided are a method, system, and article of manufacture for using a data mining algorithm to discover data rules. A data set including multiple records is processed to generate data rules for the data set. Each record has a record format including a plurality of fields and each rule provides a predicted condition for one field based on at least one predictor condition in at least one other field. The generated data rules are provided to a user interface to enable a user to edit the generated data rules. The data rules are stored in a rule repository to be available to use to validate data sets having the record format.

The invention discloses an abnormal power consumption detection method. The method comprises the following steps: A, pre-processing the currently collected data and the history data; B, detecting the pre-processed data by adopting a data mining algorithm so as to recognize the suspected abnormal power consumption users; C, taking part of the suspected abnormal power consumption users that exceed a preset threshold parameter as abnormal power consumption users to carry out feedback; D, carrying out association analysis on the abnormal power consumption data in the history data by using an association algorithm so as to extract an abnormal power consumption internal association rule, and expressing the analysis result by using a form or a graph; and E, carrying out statistics on the distribution of the abnormal power consumption users in different industries and types and the data of specific abnormal power consumption users for query. According to the abnormal power consumption method, the power consumption data accumulated by the existing power grid information acquisition system is analyzed so that the abnormal power consumption users are detected; and through detecting the abnormal power consumption users, the detection result is more comprehensive, the detection correctness and the detection efficiency are improved, and the detection time is saved.

Automated data mining using domain-specific analytic applications for solving predefined problems, including populating input data schema, the input data schema having a format appropriate to solution of a predefined problem. Production training a predefined data mining model to produce a trained data mining model, the predefined data mining model comprising a predefined data mining model definition, production training having an output of a knowledge base. Executing a preselected data mining algorithm in production training mode. Production scoring input data from the input data schema. The method typically includes scheduling the steps of populating input data schema, production training, and production scoring. Typically the analytic application includes predefined problems, predefined data mining algorithms, predefined data schema, and at least one predefined data mining model definition.

The present invention relates to a system and method for performing vehicle onboard analysis on the data associated with the vehicle and implementing a cloud-based distributed data stream mining algorithm for detecting patterns from vehicle diagnostic and correlating the pattern with the contextual data. The system applies the distributed data mining algorithms for mining the results of the vehicle onboard analytics sent over the wireless network to the server and correlates the analyzed data with the contextual data of the vehicle. The system extracts performance patterns from data, builds predictive models from vehicle diagnostic, and correlates the predicted model with the business process using state of the art link analysis techniques.

The invention provides an OLAP multi-dimensional analysis and data mining system. The system comprises a data model, a distributed OLAP engine, an OLAP analysis engine, a multi-dimensional report analysis interface, a data mining interface and a data visualization tool; the data model is dragged by a user through a visual interface to finish data modeling, and has unified model configuration; thesystem automatically performs model adaptation and enables the data model to be called by cooperation with internal other engines or tools; the distributed OLAP engine provides a multi-dimensional data model preprocessing capability for the OLAP system; the OLAP analysis engine supports a multi-dimensional query analysis engine of a big data platform and a relational database, and analyzes an MDXstatement to obtain a standard SQL; the multi-dimensional report analysis interface and the data mining interface have multi-dimensional data analysis and data mining functions and provide a report analysis method and a data mining algorithm model; and the data visualization tool provides a visual service for report analysis and data mining in the multi-dimensional report analysis interface and the data mining interface, and provides visual result social sharing and chart management functions.

The present invention provides a Spark based high efficiency text classification method. The method comprises: constructing an HDFS file system with a virtual machine and a Spark platform on a physical server, and uploading a data set into the HDFS file system; enabling the Spark platform to read data from the HDFS file system, and converting the data into RDD and storing the RDD into a memory; dividing all tasks into different stages, and then running each task; preprocessing the RDD; performing training; and testing a classification model. The method provided by the present invention makes up the defects of a naive Bayes model and further improves the processing speed; the method also effectively promotes data mining and machine learning and promotes conversion from a conventional data mining algorithm to a parallel data mining algorithm; the method improves classification precision of improving the Bayes algorithm; the method promotes improvement of a Spark platform based algorithm; and finally, the method improves cluster resource utilization.

The invention discloses a cluster sub-health early warning method and system. The method comprises the following steps of: obtaining historical operation data of a cluster; carrying out training and modeling according to the historical operation data of the cluster so as to generated a prediction model; obtaining real operation data of the cluster; taking the real-time data as an input and inputting the real-time data into the prediction model to carry out calculation so as to generate a prediction result; and judging whether the prediction result is located in a sub-health state or not, and generating an early warning signal to carry out warning when the prediction result is located in the sub-health state. According to the cluster sub-health early warning method and system, a data mining algorithm is applied to cluster operation log analysis through training and modeling, the prediction model is generated through carrying out training and modeling on the history data, and the real-time operation data is used as model input so as to predict the health condition of the cluster, so that the potential risks of the cluster can be predicted and the operation and maintenance personnel can be timely informed to carry out related processing before abnormality occurs, and then the heavy loss caused by cluster abnormality is reduced.

The invention relates to an extracting method of fire generator assemblage operation optimality principle on the basis of data mining, which utilizes real-time data bank, adopts data mining algorithm, finds out optimum operating state under the similar generator assemblage operation working order according to performance index such as stability, economy, environment protection and the like, and establishes a non-linear stable steady-state model between regulating capacity and runnability of the generator assemblage under the different disturbing capacities of the generator assemblage through data accumulation. The invention provides a real-time operation guide for operating staffs, which achieves the a purpose of optimizing fire generator assemblage operation.

The invention relates to an intelligent medical treatment personalization recommendation system and an implementation method thereof. The intelligent medical treatment personalization recommendation system is composed of a wireless sensing network and a personalization recommendation platform. The wireless sensing network acquires current key physical indexes of a patient and sends the current key physic indexes to the platform through a wireless network, and the patient inputs relevant symptoms on the platform. The personalization recommendation platform is a framework based on a B / S three-layer mode. The personalization recommendation platform analyzes former physical indexes of patients treated by doctors stored in a database based on the data acquired by the sensor and the data input by the patient, and excavates and analyzes treatment information of other patients similar with the symptoms of the patient and the former treatment records of the patient per se by using a data mining algorithm so as to provide personalization recommendation to the current patient. The patient can send an email to make an appointment based on a recommendation result.

The invention opens a data mining system based on Web, which mainly includes EJB server, Web server and database etc. The EJB server provides the interface between Web server and EJB layer, and implemented various data mining algorithms for different data and data mining tasks. The Web layer provides a user interactive interface to receive user inputs and display results of data mining and analysis. The system consists of several modules: authentication module, initialization module, data connection module, data visualization module, data pre-processing module, mining module, mining data indication module. The system provides on-line internet based data mining and result analysis services.

A computer-implemented simulator models the entire analytic value chain so that data generation, model fitting and strategy optimization are an integral part of the simulation. Data collection efforts, data mining algorithms, predictive modeling technologies and strategy development methodologies define the analytic value chain of a business operation: data→models→strategies→profit. Inputs to the simulator include consumer data and potential actions to be taken regarding a consumer or account. The invention maps what is known about a consumer or an account and the potential actions that the business can take on that consumer or account to potential future financial performance. After iteratively performing simulations using varying inputs, modeling the effect of the innovation on a profit model, the simulator outputs a prediction of the commercial value of an analytic innovation.

The present invention provides a power grid operating data processing method. The method comprises carrying out confluence analysis and visualized result display on multi-source data operated by a power grid through a data mining algorithm; and providing a configurable data drawing list designer and an issue platform used for issuing data analysis results for users. According to the power grid operating data processing method provided by the present invention, data can be dynamically acquired, related statistical confluence analysis of the data is carried out, and the data style is diversified.

The invention discloses a Hadoop-based fast neighborhood rough set attribute reduction method. The method comprises the following steps: a, establishing a distributed platform based on the Hadoop; b, defining a neighborhood rough set; c, generating a candidate set; d, calculating the importance of each attribute; e, selecting the attribute with the largest importance and adding the attribute into the candidate set; f, judging whether a stop condition is met or not; g, storing conditions selected by characteristics. The method is based on the Hadoop distributed platform to analyze the parallelization of a parallel data mining algorithm so as to realize the parallelization of a neighborhood rough set attribute reduction algorithm; the time complexity of the parallelized attribute reduction is greatly lowered, the output of an intermediate result in the performing intermediate process is greatly reduced, and the analysis efficiency of large-scale data is improved, so that numerous and varied mass data are converted into available data with information and business values, thereby completing mining and analysis optimizing of data.

The invention provides a data mining based intrusion detection system with self-learning and classified early warning functions. The system comprises a clustering analysis module, an anomaly detection engine, a rule base, a correlation analysis module, a rule generalization module, a rule management module, a log record and a classified early warning module. The data mining based intrusion detection system has the advantages that a data mining technique is applied to intrusion detection, and existing data mining algorithms and network attack characteristics are utilized fully, so that self-learning and classified early warning of the intrusion detection system are realized, detection accuracy and efficiency are improved effectively and substantial economic value and use value are achieved.

The invention relates to a data mining-based method for analyzing a thermal power plant operation index optimal target value. Operation working conditions of a unit are divided according to external working conditions of the unit operation, wherein the working conditions include loads, coal qualities, and circulating water temperatures and the like; on the basis of unit operation massive historical data accumulated by a thermal power plant, by employing a data mining algorithm, operation optimal values of all important parameters on the similar operation working conditions of the unit are found out according to performance indexes including stability, economy and environmental protection and the like; and on the basis of data accumulation and continuous data updating, operation optical values of all operation parameters of the unit are found out and tracked on different working conditions of the unit. And moreover, a concrete method according to which a thermal power plant operation index optimal value is employed as a target value of an evaluation index so as to realize operation evaluation and real-time guidance functions is provided.

The present invention discloses a method and apparatus for mining massive intelligent power consumption data based on cloud computing. The method comprises the following steps of: storing massive power consumption data generated by a peripheral system in a distributed file system; a user actively initiating a service request, and a master node receiving the request and analyzing the service request, selecting slave nodes required to participate in mining and a mining algorithm according to an actual situation, and assigning tasks to the slave nodes after decomposition of dimension; and each slave node according to the assigned task, performing data storage and task execution, using the data mining algorithm selected by master node to perform a power consumption data mining task independently, and interacting with task management. The apparatus comprises a data management module, a task management module, a task execution module, a data storage module, a mining model library module and a data dimension model module. According to the method and apparatus for mining massive intelligent power consumption data based on cloud computing, power consumption information of massive users is efficiently mined, and forecast of power consumption of domestic consumers is achieved, so as to develop an optimal power consumption strategy.

The invention provides a network attack behavior detection method. The method comprises the following steps: firstly, acquiring domain name system resolution data; secondly, performing data mining on the domain name system resolution data by a preset data mining algorithm to obtain a data mining result; and lastly, detecting network attack behaviors according to the data mining result. Compared with the conventional way of detecting the network attack behaviors by packet capturing analysis, the method has the advantages that the domain name system resolution data are taken as a processing object, and data mining is performed on the domain name system resolution data, so that the network attack behaviors can be detected more efficiently and accurately according to the mining result.

Provided are an article of manufacture, system, and method for using a data mining algorithm to generate rules used to validate a selected region of a predicted column. A data set has a plurality of columns and records providing data for each of the columns. Selection is received of at least one predicted column for which rules are to be generated and at least one region of the selected at least one predicted column, wherein each region specifies data positions in the column. The data set is processed to determine association relationships among data in at least one predictor column and subsequences in the selected at least one region of the at least one predicted column. At least one rule is generated from the relationships specifying a condition involving at least one predictor column that predicts at least one value in the selected region of the at least one predicted column.

The invention discloses a video recommendation method and system based on Web mining. The method comprises the steps that a data mining algorithm is applied in clicking behavior data when users watch videos through Web mining, a user interest model is built through a classification and regression tree, a traditional collaborative filtering algorithm is adopted to recommend an individualized video to the users, the defect that in a traditional recommendation system, the data sparsity is brought due to the fact that user comment information is little is overcome, the problem of recommendation cold start due to the fact that a new user or a new project has no scores is solved, the satisfaction degree of the users to watch the video is improved, the users having the same interest and hobbies generate a recommendation, and friend recommendation is achieved in the video recommendation system.

Automated data mining using domain-specific analytic applications for solving predefined problems, including populating input data schema, the input data schema having a format appropriate to solution of a predefined problem. Production training a predefined data mining model to produce a trained data mining model, the predefined data mining model comprising a predefined data mining model definition, production training having an output of a knowledge base. Executing a preselected data mining algorithm in production training mode. Production scoring input data from the input data schema. The method typically includes scheduling the steps of populating input data schema, production training, and production scoring. Typically the analytic application includes predefined problems, predefined data mining algorithms, predefined data schema, and at least one predefined data mining model definition.

A method of running a query for a database having partitioned tables. The method includes loading data into a table partition; forecasting statistics for the table partition based on previously gathered partition statistics using a data mining algorithm; and subsequently to forecasting statistics, running a query by a query optimizer; and wherein the method is performed by one or more computing devices. Also disclosed is a computer program product and a system.

Provided are a method, system, and article of manufacture for using a data mining algorithm to generate format rules used to validate data sets. A data set has a plurality of columns and records providing data for each of the columns. Selection is received of at least one format column for which format rules are to be generated and selection is received of at least one predictor column. A format mask column is generated for each selected format column. For records in the data set, a value in the at least one format column is converted to a format mask representing a format of the value in the format column and storing the format mask in the format mask column in the record for which the format mask was generated. The at least one predictor column and the at least one format mask column are processed to generate at least one format rule. Each format rule specifies a format mask associated with at least one condition in the at least one predictor column.

Development of domain-specific analytic applications having predefined data mining models, including identifying a business problem to be solved; selecting a data mining algorithm appropriate for solving the business problem; defining data schema for use as inputs and outputs to and from the mining algorithm, the data schema including input data schema and output data schema; and defining a mining model based on the data schema to solve the business problem; whereby a domain-specific analytic application is developed, the analytic application having at least one predefined data mining model, the analytic application having the capabilities of: populating the input data schema with sample historical data; production training the mining model using the sample historical data in the input data schema, whereby a production trained mining model is created; and production scoring production data using the production trained mining model, production scoring further comprising applying the production trained mining model to production data.