Automatic algorithm feature processing method and system

Abstract

The invention discloses an automatic algorithm feature processing method and system, and relates to the technical field of computer data processing. Data are collected and cleaned, then feature parameters are extracted, test cases are established based on the feature parameters, then a model is trained according to the feature parameters, test cases of online data feature parameters are established, and finally the test cases are input into a target server for testing, so that the whole test process is automatically carried out, and the test efficiency is effectively improved; and the testing process is suitable for artificial intelligence algorithm testing of various processing classes, and the universality is high. According to the automatic algorithm feature processing method and system, machine learning algorithms and deep learning algorithms are integrated, uniform file configuration and configurability are supported, when the data set reaches a certain amount, the deep learning strategy can be triggered, and the defect that the small data set modeling effect is poor is overcome.

Description

Technical Field

[0001] The present invention relates to the technical field of computer data processing, and in particular to an automatic algorithm feature processing method and system. Background Art

[0002] Most companies rely on internal teams of analysts and / or external contractors, as well as modelers to conduct monthly data analysis and ultimately report on customer business and product base to monitor profits, business growth or decline, and product inventory issues. The current business analysis and reporting methods are not automated, nor can they adapt to the ever-changing environment, require human resources with years of business data analysis experience, and are prone to human errors.

[0003] The invention with publication number CN114971524A discloses an automated algorithm feature processing method and system, including: acquiring business data and performing data cleaning on the business data; performing statistical analysis on the cleaned business data to obtain business variables; and graphically processing the business variables to generate customer business reports.

[0004] As shown in the above invention, the existing automated algorithm feature processing method and system can automatically analyze data to reduce personnel costs, ensure the accuracy of reports and eliminate human errors through monitoring, provide comprehensive, reliable and accurate reports to monitor the business strategy planning required for daily operation data, and can flexibly modify product business to adapt to the rapidly changing business environment. However, the feature processing process in the prior art is usually carried out according to a fixed linear processing flow. However, this fixed linear processing flow cannot adapt well to the characteristics of different tasks and different training data. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present invention provides an automated algorithm feature processing method and system to solve the existing problems.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: an automated algorithm feature processing method, comprising the following steps:

[0007] S1: Collect learning data and online data, and perform data cleaning;

[0008] S2: Extract the feature vectors of each data and construct the final feature parameters;

[0009] S3: training a model according to the characteristic parameters corresponding to the learning data, and establishing test cases of the characteristic parameters of the online data according to the best model, and using the obtained multiple test cases as a target test set;

[0010] S4: inputting multiple test cases in the target test set into the target server in sequence for testing to obtain a first test result;

[0011] S5: Perform statistical analysis on the first test result to obtain target indicator data, where the target indicator data is used to reflect the processing effect of the processing type artificial intelligence algorithm.

[0012] Preferably, the target test set includes a plurality of test subsets, wherein a confusion matrix is ​​used to classify a plurality of test cases to obtain a plurality of test subsets, wherein each test subset includes test cases of the same type.

[0013] Preferably, multiple test cases are sent to the target server in sequence, and the target server is used to call an artificial intelligence algorithm of the processing type to process the input multiple test cases in sequence; and receive a first test result fed back by the target server.

[0014] Preferably, the training models include a naive Bayes model, a logistic regression model, a decision tree model, a random forest model, an XGBoost model, and a recurrent neural network model, and the best model is determined by predetermined indicators.

[0015] Preferably, data cleaning is performed on the first test result to obtain a second test result; and a confusion matrix is ​​used to perform statistical analysis on the second test result to obtain target indicator data, wherein the type of the target indicator data includes at least one of the following: accuracy, precision, recall, a balanced comprehensive indicator of precision and recall, false negative rate, true negative rate, and false positive rate.

[0016] Preferably, after obtaining the target indicator data, the plurality of test results and the target indicator data are stored in a target location, wherein the target location includes at least one of the following: a cloud server, a target database.

[0017] Preferably, after obtaining the target indicator data, when the target indicator data does not meet the preset indicator requirements, a prompt message is sent to the target object, and the prompt message is used to prompt the target object to train the processing-type artificial intelligence algorithm again.

[0018] The present invention also discloses an automated algorithm feature processing system, comprising:

[0019] Data collection module, used to collect learning data and online data;

[0020] Feature extraction module, used to extract the feature vectors of each data and construct the final feature parameters;

[0021] A training module, for training a model according to characteristic parameters corresponding to the learning data;

[0022] Establish a module to establish test cases for online data feature parameters based on the best training model;

[0023] A target server, used to test multiple test cases in a target test set;

[0024] The analysis module is used to perform statistical analysis on the test results to obtain target indicator data.

[0025] Preferably, the system further comprises a background scheduler, and the background scheduler triggers the data acquisition module according to a predetermined time period.

[0026] Preferably, the target server is a single or multiple servers or server clusters carrying a target interface, and the target interface is one or more interfaces encapsulating a processing-type artificial intelligence algorithm.

[0027] Beneficial Effects

[0028] The present invention provides an automated algorithm feature processing method and system. Compared with the prior art, it has the following beneficial effects:

[0029] 1. The automated algorithm feature processing method and system collects and cleans data, extracts feature parameters, and establishes test cases based on the feature parameters. Then, the model is trained according to the feature parameters, and test cases for the feature parameters of online data are established. Finally, the test is input into the target server for testing. The entire test process is automated, which effectively improves the test efficiency. The test process is suitable for testing artificial intelligence algorithms of various processing types and has strong versatility.

[0030] 2. This automated algorithm feature processing method and system integrates various machine learning algorithms and deep learning algorithms, supports a unified configuration file, and is configurable. When the data set reaches a certain amount, it can trigger a deep learning strategy, solving the defect of poor modeling effect of small data sets. In terms of machine learning algorithms, it integrates multiple algorithms and selects the best algorithm model in an automated manner. It can also expand the algorithm by itself, thereby solving the algorithm selection problem to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a schematic diagram of the structure of the method flow of the present invention;

[0032] Figure 2 It is a schematic diagram of system connection of the present invention. DETAILED DESCRIPTION

[0033] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0034] In order to better understand the embodiments of the present application, some nouns or terms that appear in the description of the embodiments of the present application are first translated and explained as follows:

[0035] Test Case: A test case is a description of a test task for a specific software product, reflecting the test plan, method, technology and strategy. Its content includes test objectives, test environment, input data, test steps, expected results, test scripts, etc., and finally forms a document. Simply put, a test case is a set of test inputs, execution conditions and expected results compiled for a special goal, which is used to verify whether a specific software requirement is met. Usually, a test case mainly contains four contents: test case title, preconditions, test steps and expected results. Among them, the test case title mainly describes the test of a certain function; the precondition refers to the condition that the test case title needs to meet; the test steps mainly describe the operation steps of the use case; the expected result refers to the compliance with the expected requirements (development specifications, requirement documents, user requirements, etc.).

[0036] Confusion Matrix: Also known as the error matrix, it is a standard format for expressing accuracy evaluation, expressed in the form of a matrix with n rows and n columns. Specific evaluation indicators include overall accuracy, mapping accuracy, user accuracy, etc. These accuracy indicators reflect the accuracy of image classification from different aspects. In artificial intelligence, the confusion matrix is ​​a visualization tool, especially used for supervised learning, and is generally called a matching matrix in unsupervised learning. In image accuracy evaluation, it is mainly used to compare classification results with actual measured values. The accuracy of classification results can be displayed in a confusion matrix. The confusion matrix is ​​calculated by comparing the position and classification of each measured pixel with the corresponding position and classification in the classified image.

[0037] See Figure 1-2 , the present invention provides the following two technical solutions:

[0038] The first implementation method: an automated algorithm feature processing method, comprising the following steps:

[0039] S1: Collect learning data and online data, and perform data cleaning;

[0040] S2: Extract the feature vectors of each data and construct the final feature parameters;

[0041] S3: training a model according to the characteristic parameters corresponding to the learning data, and establishing test cases of the characteristic parameters of the online data according to the best model, and using the obtained multiple test cases as a target test set;

[0042] S4: inputting multiple test cases in the target test set into the target server in sequence for testing to obtain a first test result;

[0043] S5: Perform statistical analysis on the first test result to obtain target indicator data, where the target indicator data is used to reflect the processing effect of the processing type artificial intelligence algorithm.

[0044] In specific implementation, the target test set includes multiple test subsets, wherein a confusion matrix is ​​used to classify multiple test cases to obtain multiple test subsets, wherein each test subset includes test cases of the same type, and the multiple test cases are sent to the target server in sequence, and the target server is used to call an artificial intelligence algorithm of the processing class to process the input multiple test cases in sequence; and receive a first test result fed back by the target server.

[0045] A variety of algorithm models are established in the training model, including a naive Bayes model, a logistic regression model, a decision tree model, a random forest model, an XGBoost model, and a recurrent neural network model, and the best model is determined by predetermined indicators.

[0046] During specific implementation, data cleaning is performed on the first test result to obtain a second test result; a confusion matrix is ​​used to perform statistical analysis on the second test result to obtain target indicator data, wherein the type of the target indicator data includes at least one of the following: accuracy, precision, recall, a balanced comprehensive indicator of precision and recall, false negative rate, true negative rate, and false positive rate; after obtaining the target indicator data, multiple test results and target indicator data are stored in a target location, wherein the target location includes at least one of the following: a cloud server, a target database; after obtaining the target indicator data, when the target indicator data does not meet the preset indicator requirements, a prompt message is sent to the target object, and the prompt message is used to prompt the target object to re-train the processing-type artificial intelligence algorithm.

[0047] Second implementation mode: The present invention also discloses an automated algorithm feature processing system, comprising:

[0048] Data collection module, used to collect learning data and online data;

[0049] Feature extraction module, used to extract the feature vectors of each data and construct the final feature parameters;

[0050] A training module, for training a model according to characteristic parameters corresponding to the learning data;

[0051] Establish a module to establish test cases for online data feature parameters based on the best training model;

[0052] A target server, used to test multiple test cases in a target test set;

[0053] The analysis module is used to perform statistical analysis on the test results to obtain target indicator data.

[0054] During specific implementation, the system also includes a background scheduler, which triggers a data acquisition module according to a predetermined time period. The data acquisition module collects learning data and online data, performs data cleaning through this system to obtain relatively pure data, and then inputs the data into a feature extraction module for feature extraction to obtain respective feature vectors, and then performs feature processing on the feature vectors to obtain final feature parameters. According to the feature parameters after feature extraction, a suitable algorithm model is selected through the training module of the integrated strategy algorithm for model training, and then a test case for the online data feature parameters is established according to the establishment module. Finally, statistical analysis is performed through the analysis module to obtain target indicator data.

[0055] In an optional instance, the data acquisition module is preset with a default cleaning function and a custom cleaning function interface, and the data is cleaned by the default cleaning function or the custom cleaning function. The feature extraction module obtains different feature values ​​by configuring multiple behavior recognition functions, and the behavior recognition function includes a hot coding behavior recognition function, a weighted behavior recognition function, and a conventional numerical behavior recognition function. The feature extraction module is also provided with a default cleaning function and a custom cleaning function interface, and the data feature values ​​are cleaned by the default cleaning function or the custom cleaning function, wherein the default cleaning function includes a deduplication filter function and a missing value filter function.

[0056] In an optional example, the system can also set up a data cleaning channel, which provides a default cleaning function and a custom cleaning function, and opens an extension interface to the outside world, mainly to filter out some special or unnecessary features;

[0057] The behavior function recognition channel is the key to feature extraction. Its purpose is to convert a string into a numerical value that needs to be calculated by the subsequent algorithm. The default behavior recognition functions provided are: hot coding behavior recognition function, weighted behavior recognition function, and conventional numerical behavior recognition function. At the same time, this channel also supports externally open extension interfaces. Finally, the unified processor of this channel integrates it. It only needs to pass in the corresponding parameters provided to select the corresponding behavior recognition function for different feature dimensions.

[0058] The feature vector channel collects the output data of the behavior function recognition channel of each node and generates the corresponding feature vector according to each node;

[0059] The feature processing channel collects the output data of the feature vector channel respectively, normalizes, reduces noise and performs correlation processing on each node data to achieve a relatively pure node vector, and provides an external expansion interface;

[0060] Vector set channel, merges the vectors of each feature processing channel, generates the final feature parameters, and prepares for modeling.

[0061] During the specific implementation, a variety of algorithm models were established in the training model, including the naive Bayes model, logistic regression model, decision tree model, random forest model, XGBoost model, and recurrent neural network model, and the optimal model was determined through predetermined indicators.

[0062] In specific implementation, the data branch can be determined through the background activation mechanism vul_activate function, and the data can be pushed to the input_piplines pipeline, the data can be cleaned in the feature_clean_module, and then pushed to the vectorize_module. Finally, the data flows into the feature_extract_piplines pipeline to prepare for modeling; strategy modeling is carried out according to the build_module, and the established model is pushed to the risk_evaluate_module to obtain the target indicator data.

[0063] In an optional example, the training module also includes a reinforcement learning module, which includes TreeNode, MCTS, PolicyValueNet, Sequence, and Search_Sequence. The specific role of each module will be introduced below.

[0064] reeNode: represents a node in the Monte Carlo tree. The attributes of this class mainly include parent and child nodes, node visit times, node value Q, and U used to calculate the upper limit of UCB. The node implements the three methods of selection, expansion, and reverse update in the Monte Carlo tree search process.

[0065] MCTS: Provides the overall control flow of Monte Carlo Tree Search.

[0066] There are three main methods:

[0067] (1) get_move_probs: responsible for controlling the simulation process and calling the playout method multiple times.

[0068] (2) play_out: simulation process. Each playout expands and selects nodes from the current state. When the simulation termination condition is reached, the node value is reversely updated based on the statistical information. After multiple simulations, the probability of each action corresponding to the state can be obtained based on the number of node visits.

[0069] (3) update_with_move: Checks whether the action selected at this time is in the child node. If so, sets the child node corresponding to the action as the new root node.

[0070] PolicyValueNet: This class is mainly responsible for outputting the corresponding action strategy and evaluation value according to the input state, and training the policy value neural network through the data of the previous round after each round of search.

[0071] Method Name Method Description policy_value Returns the policy and evaluation value policy_valuefn Returns policy, action and evaluation value train_step Training the Policy Value Network get_policy_params Returns the policy value network parameters save_model Save the model

[0072] Table 1 PolicyValueNet class methods

[0073] Sequence: This class has three main functions. The first is to obtain the corresponding encoding space according to the data set information and then initialize the state. The second is to define the feature transformation function sequence as a state, and then perform high-value actions in the state to form a new state. Finally, before evaluating the feature transformation function sequence, it is responsible for decoding the state, controlling the feature transformation execution process, and calling the evaluation function to evaluate the new data set.

[0074] Method Name Method Description init_sequence Initialize sequence to empty move_to_location Reading of data sets and data information location_to_move Set the feature transformation function search space current_state Read user-defined classes based on unique class names do_move Setting up the evaluation model search_end Setting model hyperparameters state_evaluate Setting the evaluation method

[0075] Table 2 Sequence class methods

[0076] Search_Sequence: The high-level abstract class of the reinforcement learning module, which is mainly responsible for controlling the operation process of the entire subsystem.

[0077] Meanwhile, the contents not described in detail in this specification belong to the prior art known to those skilled in the art, and the model parameters of each electrical appliance are not specifically limited, and conventional equipment can be used.

[0078] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0079] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An automated algorithm feature processing method, characterized in that: The following steps are involved: S1: Collect learning data and online data, and perform data cleaning; S2: Extract the feature vectors of each data and construct the final feature parameters; S3: training a model according to the characteristic parameters corresponding to the learning data, and establishing test cases of the characteristic parameters of the online data according to the best model, and using the obtained multiple test cases as a target test set; S4: inputting multiple test cases in the target test set into the target server in sequence for testing to obtain a first test result; S5: Perform statistical analysis on the first test result to obtain target indicator data, where the target indicator data is used to reflect the processing effect of the processing type artificial intelligence algorithm.

2. The method of automatic algorithm feature processing according to claim 1, characterized in that: The target test set includes multiple test subsets, wherein multiple test cases are classified using a confusion matrix to obtain multiple test subsets, wherein each test subset includes test cases of the same type.

3. The automated algorithm feature processing method according to claim 1, characterized in that: Sending multiple test cases to the target server in sequence, and the target server is used to call the artificial intelligence algorithm of the processing class to process the input multiple test cases in sequence; Receive a first test result fed back by the target server.

4. The automated algorithm feature processing method according to claim 1, characterized in that: The training models include a naive Bayes model, a logistic regression model, a decision tree model, a random forest model, an XGBoost model, and a recurrent neural network model, and the best model is determined by predetermined indicators.

5. The method of automatic algorithm feature processing according to claim 1, characterized in that: The first test result is cleaned to obtain a second test result; the second test result is statistically analyzed using a confusion matrix to obtain target indicator data, wherein the type of the target indicator data includes at least one of the following: accuracy, precision, recall, a balanced comprehensive indicator of precision and recall, false negative rate, true negative rate, and false positive rate.

6. The method of automatic algorithm feature processing according to claim 5, characterized in that: After obtaining the target indicator data, the plurality of test results and the target indicator data are stored in a target location, wherein the target location includes at least one of the following: a cloud server, a target database.

7. The automated algorithm feature processing method according to claim 5, characterized in that: After obtaining the target indicator data, when the target indicator data does not meet the preset indicator requirements, a prompt message is sent to the target object, and the prompt message is used to prompt the target object to train the processing-type artificial intelligence algorithm again.

8. An automated algorithm feature processing system, characterized in that: include: Data collection module, used to collect learning data and online data, and perform data cleaning; Feature extraction module, used to extract the feature vectors of each data and construct the final feature parameters; A training module, for training a model according to characteristic parameters corresponding to the learning data; Establish a module to establish test cases for online data feature parameters based on the best training model; A target server, used to test multiple test cases in a target test set; The analysis module is used to perform statistical analysis on the test results to obtain target indicator data.

9. The automated algorithm feature processing system according to claim 8, characterized in that: The system also includes a background scheduler, which triggers the data acquisition module according to a predetermined time period.

10. The automated algorithm feature processing system according to claim 8, characterized in that: The target server is a single or multiple servers or server clusters carrying target interfaces, and the target interface is one or more interfaces encapsulating processing-type artificial intelligence algorithms.

Images