Method, system, equipment and medium for building artificial intelligence model based on big data

Abstract

The present application provides a method, system, device and medium for building an artificial intelligence model based on big data. The method according to the present application includes: deploying an artificial intelligence model building library in a server, wherein the artificial intelligence model component library includes model building elements and a model finished product library; judging whether there is a corresponding target artificial intelligence model in the model finished product library according to the construction requirements of the target artificial intelligence model, and directly calling the target artificial intelligence model if it exists, otherwise generating a model building element set, and generating instruction information for building the target artificial intelligence model according to the model building element set; the server parses and determines the configuration resources required to build the target artificial intelligence model in response to the instruction information, and performs distributed training based on the configuration resources to generate a trained target artificial intelligence model. The technical solution provided by the present application builds an artificial intelligence model through a model building element set, thereby improving modeling efficiency and lowering the modeling threshold.

Description

Technical Field

[0001] This document relates to the technical field of artificial intelligence model construction, and in particular to a method, system, device and medium for building an artificial intelligence model based on big data. Background Art

[0002] With the rapid advancement of science and technology, intelligence has become the mainstream trend of today's social development. The core of this trend lies in the automated processing of data, which is inseparable from the strong support of various artificial intelligence models. Artificial intelligence models, as cutting-edge technologies in the fields of mathematics, statistics, computer science and machine learning, are mathematical models that deeply analyze, accurately process, scientifically predict and optimize regular and predictable data, and have demonstrated their unparalleled value in many fields.

[0003] In key areas such as computer vision, image processing, natural language processing, information classification, search recommendation, and big data, artificial intelligence models are playing an unprecedented role in promoting the development of these areas. They not only greatly improve the processing efficiency and accuracy of these areas, but also bring us unprecedented convenience and intelligent experience.

[0004] However, in the process of building these artificial intelligence models, we also have to face some challenges. First, the construction of the model requires technical personnel with deep professional knowledge to write code, which not only leads to high usage costs, but also makes the model training cycle long and complicated. Secondly, the existing models often lack universality and are difficult to meet the personalized needs of various users, which further raises the threshold of AI modeling and makes it difficult for ordinary users to get involved in this field.

[0005] In order to solve these problems, we need to continue to explore and innovate to lower the threshold of AI modeling and improve the versatility and ease of use of the model. Only in this way can we make artificial intelligence models better serve society and bring convenience and intelligence to more people. Summary of the invention

[0006] The embodiments of the present invention provide a method, system, device and medium for building an artificial intelligence model based on big data, aiming to solve the above-mentioned problems.

[0007] According to an embodiment of the present invention, a method for building an artificial intelligence model based on big data is provided, comprising:

[0008] Deploy an artificial intelligence model building library in the server, wherein the artificial intelligence model component library includes model building elements and a model finished product library;

[0009] According to the construction requirements of the target artificial intelligence model, determine whether there is a corresponding target artificial intelligence model in the model finished product library, and if so, directly call the target artificial intelligence model; otherwise, generate a model construction element set, and generate instruction information for constructing the target artificial intelligence model according to the model construction element set;

[0010] The server parses the instruction information in response to determine the configuration resources required to build the target artificial intelligence model, performs distributed training based on the configuration resources, and generates a trained target artificial intelligence model.

[0011] According to an embodiment of the present invention, a system for building an artificial intelligence model based on big data is provided, comprising:

[0012] An element deployment module deploys an artificial intelligence model building library in a server, wherein the artificial intelligence model component library includes model building elements and a model finished product library, wherein the model building elements include: industrial data set elements, feature selection elements, data segmentation elements, model algorithm elements, model tuning elements, and verification elements;

[0013] An instruction generation module determines whether there is a corresponding target artificial intelligence model in the model finished product library according to the construction requirements of the target artificial intelligence model. If so, the target artificial intelligence model is directly called; otherwise, a model construction element set is generated, and the model construction elements in the model construction element set are spliced ​​to generate instruction information for constructing the target artificial intelligence model;

[0014] The model generation module, the server parses the instruction information in response to determine the configuration resources required to build the target artificial intelligence model, performs distributed training based on the configuration resources, and generates a trained target artificial intelligence model.

[0015] According to an embodiment of the present invention, there is further provided an electronic device, including:

[0016] processor; and,

[0017] A memory arranged to store computer executable instructions, which, when executed, cause the execution of the steps of the above-mentioned method for building an artificial intelligence model based on big data.

[0018] According to an embodiment of the present invention, a storage medium is also provided for storing computer executable instructions, which, when executed, implement the steps of the above-mentioned method for building an artificial intelligence model based on big data.

[0019] According to an embodiment of the present invention, each part of the artificial intelligence algorithm model construction is encapsulated one by one as the corresponding model construction element, so that the user can splice each model element by dragging and dropping, generate instruction information for building the target artificial intelligence model, parse in response to the instruction information, build the configuration resources required for the target artificial intelligence model according to the model construction elements, perform distributed training based on the configuration resources, and generate a trained target artificial intelligence model. Through the above method, it is possible to use graphical operations and use components to build AI models, which has a low threshold and is easy to build without writing code, greatly improving the modeling efficiency of artificial intelligence models; on the other hand, through component graphical modeling, not only are the node processes clear and understandable, but also easy to understand. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate one or more embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0021] Figure 1 This is a flowchart of a method for building an artificial intelligence model based on big data according to an embodiment of the present invention;

[0022] Figure 2 A schematic diagram of an artificial intelligence model building system based on big data according to an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the following will be combined with the drawings in one or more embodiments of this specification to clearly and completely describe the technical solutions in one or more embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this document.

[0024] Method Embodiment

[0025] According to an embodiment of the present invention, a method for building an artificial intelligence model based on big data is provided. Figure 1 is a flowchart of a method for building an artificial intelligence model based on big data according to an embodiment of the present invention. Figure 1 As shown, the method for building an artificial intelligence model based on big data in an embodiment of the present invention specifically includes:

[0026] S1. Deploy an artificial intelligence model building library in a server, wherein the artificial intelligence model component library includes model building elements and a model finished product library;

[0027] The model building elements include: industrial data set elements, feature selection elements, data segmentation elements, model algorithm elements, verification elements and model tuning elements.

[0028] The steps for obtaining the elements of the industrial dataset are as follows:

[0029] Collect industrial data within a preset period, pre-process the industrial data to obtain standard industrial data and determine whether the standard industrial data meets the preset conditions; if the standard industrial data meets the preset conditions, classify and store the standard industrial data.

[0030] The industrial data preprocessing specifically includes:

[0031] The acquired industrial data is cleaned to obtain cleaned industrial data; the cleaned industrial data is further time aligned to align the industrial data to a uniform time scale according to the timestamp information carried in the industrial data, and the industrial data is interpolated and completed to obtain an industrial data time series as pre-processed standard industrial data; wherein, the acquired original industrial data is cleaned, specifically including: according to the data item type of the industrial data, retrieving the data standard range corresponding to the data item type; according to the obtained data standard range, the obtained industrial data is subjected to data consistency check, invalid value correction and missing value completion to obtain the cleaned industrial data.

[0032] The model algorithm elements include:

[0033] Decision tree algorithm, Adboost algorithm, residual neural network algorithm, target detection algorithm, and word steering algorithm.

[0034] The instruction information for generating a target artificial intelligence model according to the model building element set specifically includes:

[0035] Acquire a data set for model training of a target artificial intelligence model according to the industrial data set elements in the model building element set;

[0036] Performing a feature extraction operation on the data set according to the feature selection factor;

[0037] Dividing the data set combined with the data for model training into a training set and a validation set according to a preset ratio according to the data segmentation factor;

[0038] Selecting a machine learning algorithm for model training according to the model algorithm class elements, and initializing the machine learning algorithm;

[0039] The verification element is used to verify the model output result according to the verification set to obtain the verification result;

[0040] The model tuning elements are used to adjust the parameters of the machine learning algorithm for model training according to the verification results.

[0041] S2. According to the construction requirements of the target artificial intelligence model, determine whether there is a corresponding target artificial intelligence model in the model finished product library. If so, directly call the target artificial intelligence model; otherwise, generate a model construction element set, and generate instruction information for constructing the target artificial intelligence model according to the model construction element set;

[0042] Display the model building elements in the artificial intelligence algorithm library in the tool component area in the form of text representation, and select the model building elements in the tool component area by dragging and dropping;

[0043] Move the model building elements into the modeling canvas area, and generate instruction information for building the target artificial intelligence model based on the description information and control parameters of each model building element.

[0044] S3. The server parses the instruction information in response to determine the configuration resources required to build the target artificial intelligence model, performs distributed training based on the configuration resources, and generates a trained target artificial intelligence model.

[0045] The trained target artificial intelligence model is further stored in the artificial intelligence model component library.

[0046] Optionally, the artificial intelligence model building method further includes:

[0047] Industrial data set elements, feature selection elements, data segmentation elements, model algorithm elements, verification elements, and model tuning elements

[0048] Select the industrial dataset elements that match the target artificial intelligence model in the model building elements and import them into the dataset to form the target dataset;

[0049] Extracting features from the target data set according to the feature selection factors; specifically including: preprocessing the target data set using the training class operator, the preprocessing including missing value filling, duplicate row removal, random sampling and data type conversion; performing feature processing on the preprocessed target data set, the feature processing including feature normalization, feature standardization, outlier processing, one-hot encoding data conversion and wrapped feature selection;

[0050] The target data set is segmented with the data segmentation factor and divided into a training set and a validation set according to a preset ratio;

[0051] According to the model algorithm elements, select the appropriate algorithm to train the training set and generate the model results;

[0052] Verification elements are used to verify whether the model results meet the preset requirements. If not, optimization training is performed through model tuning elements.

[0053] By adopting the embodiments of the present invention, the following beneficial effects are achieved:

[0054] According to an embodiment of the present invention, each part of the artificial intelligence algorithm model construction is encapsulated one by one as the corresponding model construction element, so that the user can splice each model element by dragging and dropping, generate instruction information for building the target artificial intelligence model, parse in response to the instruction information, build the configuration resources required for the target artificial intelligence model according to the model construction elements, perform distributed training based on the configuration resources, and generate a trained target artificial intelligence model. Through the above method, it is possible to use graphical operations and use components to build AI models, which has a low threshold and is easy to build without writing code, greatly improving the modeling efficiency of artificial intelligence models; on the other hand, through component graphical modeling, not only are the node processes clear and understandable, but also easy to understand.

[0055] System Example

[0056] According to an embodiment of the present invention, a system for building an artificial intelligence model based on big data is provided. Figure 2 Schematic diagram of a system for building an artificial intelligence model based on big data according to an embodiment of the present invention. Figure 2 As shown, the artificial intelligence model construction system based on big data in an embodiment of the present invention specifically includes:

[0057] An element deployment module 20 deploys an artificial intelligence model building library in a server, wherein the artificial intelligence model component library includes model building elements and a model finished product library, wherein the model building elements include: industrial data set elements, feature selection elements, data segmentation elements, model algorithm elements, model tuning elements, and verification elements;

[0058] The instruction generation module 21 determines whether there is a corresponding target artificial intelligence model in the model finished product library according to the construction requirements of the target artificial intelligence model. If so, the target artificial intelligence model is directly called. Otherwise, a model construction element set is generated, and the model construction elements in the model construction element set are spliced ​​to generate instruction information for constructing the target artificial intelligence model.

[0059] The model generation module 22, the server parses the instruction information to determine the configuration resources required to build the target artificial intelligence model, performs distributed training based on the configuration resources, and generates a trained target artificial intelligence model.

[0060] Device Example 1

[0061] According to an embodiment of the present invention, there is provided an electronic device, including:

[0062] processor; and,

[0063] A memory arranged to store computer executable instructions which, when executed, cause the steps of the above method embodiments to be performed.

[0064] Device Example 2

[0065] According to an embodiment of the present invention, a storage medium is provided for storing computer executable instructions, wherein the computer executable instructions implement the steps of the above method embodiment when executed.

[0066] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for building an artificial intelligence model based on big data, characterized in that include: Deploy an artificial intelligence model building library in the server, wherein the artificial intelligence model component library includes model building elements and a model finished product library; According to the construction requirements of the target artificial intelligence model, determine whether there is a corresponding target artificial intelligence model in the model finished product library, and if so, directly call the target artificial intelligence model; otherwise, generate a model construction element set, and generate instruction information for constructing the target artificial intelligence model according to the model construction element set; The server parses the instruction information in response to determine the configuration resources required to build the target artificial intelligence model, performs distributed training based on the configuration resources, and generates a trained target artificial intelligence model.

2. The method according to claim 1, characterized in that The model building elements include: industrial data set elements, feature selection elements, data segmentation elements, model algorithm elements, verification elements and model tuning elements.

3. The method according to claim 2, characterized in that The steps for obtaining the elements of the industrial dataset are as follows: Collect industrial data within a preset period, pre-process the industrial data to obtain standard industrial data and determine whether the standard industrial data meets the preset conditions; if the standard industrial data meets the preset conditions, classify and store the standard industrial data.

4. The method according to claim 3, characterized in that The industrial data preprocessing specifically includes: The acquired industrial data is cleaned to obtain cleaned industrial data; the cleaned industrial data is further time aligned to align the industrial data to a uniform time scale according to the timestamp information carried in the industrial data, and the industrial data is interpolated and completed to obtain an industrial data time series as pre-processed standard industrial data; wherein, the acquired original industrial data is cleaned, specifically including: according to the data item type of the industrial data, retrieving the data standard range corresponding to the data item type; according to the obtained data standard range, the obtained industrial data is subjected to data consistency check, invalid value correction and missing value completion to obtain the cleaned industrial data.

5. The method according to claim 2, characterized in that: The model algorithm elements include: Decision tree algorithm, Adboost algorithm, residual neural network algorithm, target detection algorithm, and word steering algorithm.

6. The method according to claim 2, characterized in that The instruction information for constructing a target artificial intelligence model generated according to the model construction element set specifically includes: Acquire a data set for model training of a target artificial intelligence model according to the industrial data set elements in the model building element set; Performing a feature extraction operation on the data set according to the feature selection factor; Dividing the data set combined with the data for model training into a training set and a validation set according to a preset ratio according to the data segmentation factor; Selecting a machine learning algorithm for model training according to the model algorithm class elements, and initializing the machine learning algorithm; The verification element is used to verify the model output result according to the verification set to obtain the verification result; The model tuning elements are used to adjust the parameters of the machine learning algorithm for model training according to the verification results.

7. The method according to claim 1, characterized in that The trained target artificial intelligence model is further stored in the artificial intelligence model component library.

8. An artificial intelligence model construction system based on big data, characterized in that: include: An element deployment module deploys an artificial intelligence model building library in a server, wherein the artificial intelligence model component library includes model building elements and a model finished product library, wherein the model building elements include: industrial data set elements, feature selection elements, data segmentation elements, model algorithm elements, model tuning elements, and verification elements; An instruction generation module determines whether there is a corresponding target artificial intelligence model in the model finished product library according to the construction requirements of the target artificial intelligence model. If so, the target artificial intelligence model is directly called; otherwise, a model construction element set is generated, and the model construction elements in the model construction element set are spliced ​​to generate instruction information for constructing the target artificial intelligence model; The model generation module, the server parses the instruction information in response to determine the configuration resources required to build the target artificial intelligence model, performs distributed training based on the configuration resources, and generates a trained target artificial intelligence model.

9. An electronic device, comprising: processor; as well as, A memory arranged to store computer executable instructions, which, when executed, cause the execution of the steps of the method for building an artificial intelligence model based on big data as described in any one of claims 1 to 7.

10. A storage medium for storing computer executable instructions, which, when executed, implement the steps of the method for building an artificial intelligence model based on big data as described in any one of claims 1 to 7.

Images