Behavior detection method, device and equipment

Abstract

The invention provides a behavior detection method, device and equipment. The method comprises the following steps: selecting a plurality of neighbor verification tables corresponding to a to-be-detected table of a target object from a verification table set; for each network model, determining an accuracy evaluation parameter of the network model based on the distance between the to-be-detected table and each neighbor verification table and a prediction label of the network model for each neighbor verification table; determining diversity evaluation parameters of the network model based on the prediction label of the network model for each neighbor verification table and the prediction labels of other network models except the network model for each neighbor verification table; and selecting a plurality of candidate network models from all the network models based on the accuracy evaluation parameter and the diversity evaluation parameter of each network model, and determining a target prediction result of the to-be-detected table based on the plurality of candidate network models, the target prediction result being used for detecting whether the target object has a specified type of behavior. Through the technical scheme of the invention, the management abnormity of the target object is avoided.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a behavior detection method, device and equipment. Background Art

[0002] With the rapid development of artificial intelligence, the application of artificial intelligence is increasing, and various industries have begun to introduce machine learning methods to solve practical problems. Tabular data is a common form of data in various industries, such as finance, energy, and medical fields. Machine learning methods are used to solve business problems in these fields.

[0003] When using machine learning methods to solve business problems, tabular data is usually input into a machine learning model, which is then used to detect whether the target object has a specified type of behavior, such as predicting whether the target object will have a specified type of behavior in the future, and then managing the target object.

[0004] However, if the accuracy of the machine learning model is poor, it will not be able to accurately predict whether the target object will have a specified type of behavior in the future, resulting in abnormal management of the target object and business failure. Summary of the invention

[0005] The present application provides a behavior detection method, which is applied to a server, wherein the server supports multiple network models. If the multiple network models are used to implement a regression task, the method includes:

[0006] Selecting a plurality of neighbor verification tables corresponding to the to-be-detected table of the target object from the verification table set;

[0007] For each network model, based on the distance between the table to be detected and each neighbor verification table and the predicted label of each neighbor verification table by the network model, determine the accuracy evaluation parameter of the network model; based on the predicted label of each neighbor verification table by the network model and the predicted label of each neighbor verification table by other network models except the network model, determine the diversity evaluation parameter of the network model;

[0008] Based on the accuracy evaluation parameter and diversity evaluation parameter of each network model, multiple candidate network models are selected from all network models, and the target prediction result of the table to be detected is determined based on the multiple candidate network models, and the target prediction result is used to detect whether the target object has a specified type of behavior.

[0009] The present application provides a behavior detection device, which is applied to a server, wherein the server supports multiple network models. If the multiple network models are used to implement a regression task, the device includes:

[0010] A selection module, used to select a plurality of neighbor verification tables corresponding to the table to be detected of the target object from the verification table set; wherein the verification table set includes a plurality of verification tables;

[0011] A determination module is used to determine, for each network model, the accuracy evaluation parameters of the network model based on the distance between the table to be detected and each neighbor verification table and the predicted label of each neighbor verification table by the network model; and the diversity evaluation parameters of the network model based on the predicted label of each neighbor verification table by the network model and the predicted label of each neighbor verification table by other network models other than the network model;

[0012] The selection module is further used to select multiple candidate network models from all network models based on the accuracy evaluation parameter of each network model and the diversity evaluation parameter of each network model;

[0013] The determination module is further used to determine a target prediction result of the to-be-detected table based on the multiple candidate network models, and the target prediction result is used to detect whether the target object has a specified type of behavior.

[0014] The present application provides an electronic device, comprising: a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the behavior detection method of the above example of the present application.

[0015] It can be seen from the above technical scheme that in the embodiment of the present application, multiple neighbor verification tables can be selected from the verification table set, and the accuracy evaluation parameters and diversity evaluation parameters of the network model can be determined based on the predicted labels of each neighbor verification table by the network model. Multiple candidate network models are selected from all network models based on the accuracy evaluation parameters and the diversity evaluation parameters, and the target prediction results of the table to be detected are determined based on the multiple candidate network models, and the target prediction results are used to detect whether the target object has a specified type of behavior. Accurately predict whether the target object has a specified type of behavior in the future time through multiple candidate network models, and then manage the target object to avoid abnormal management of the target object and avoid business failures. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments of the present application or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings of the embodiments of the present application.

[0017] Figure 1is a flow chart of a behavior detection method in one embodiment of the present application;

[0018] Figure 2 is a flow chart of a behavior detection method in one embodiment of the present application;

[0019] Figure 3 is a flow chart of a behavior detection method in one embodiment of the present application;

[0020] Figure 4 is a flow chart of a behavior detection method in one embodiment of the present application;

[0021] Figure 5 is a structural schematic diagram of a behavior detection device in one embodiment of the present application;

[0022] Figure 6 It is a hardware structure diagram of an electronic device in one embodiment of the present application. DETAILED DESCRIPTION

[0023] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, rather than limiting the present application. The singular forms of "a", "said" and "the" used in the present application and claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to any or all possible combinations of one or more associated listed items.

[0024] It should be understood that, although the terms first, second, third, etc. may be used to describe various information in the embodiments of the present application, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, in addition, the word "if" used may be interpreted as "at..." or "when..." or "in response to determination".

[0025] In the embodiment of the present application, a behavior detection method is proposed, which can be applied to a server (such as a personal computer, a terminal device, a smart phone, an IPC, an NVR, an online platform, etc., any device that supports the behavior detection function can be called a server, and the type of the server is not limited in this embodiment), and the server can support multiple network models. If multiple network models are used to implement regression tasks, see Figure 1 FIG. 1 is a flow chart of the behavior detection method, which may include:

[0026] Step 101: Select multiple neighbor verification tables corresponding to the table to be detected of the target object from the verification table set. The verification table set may include multiple verification tables (ie, verification data).

[0027] Step 102: For each network model, based on the distance between the table to be detected and each neighbor verification table, and the predicted label of each neighbor verification table by the network model, determine the accuracy evaluation parameter of the network model; based on the predicted label of each neighbor verification table by the network model, and the predicted label of each neighbor verification table by other network models except the network model, determine the diversity evaluation parameter of the network model.

[0028] Step 103: Select multiple candidate network models from all network models based on the accuracy evaluation parameters and diversity evaluation parameters of each network model, and determine the target prediction result of the table to be detected based on the multiple candidate network models. The target prediction result is used to detect whether the target object has a specified type of behavior.

[0029] Exemplarily, selecting multiple neighbor verification tables corresponding to the table to be detected of the target object from the verification table set may include but is not limited to: determining the distance between the table to be detected and each verification table in the verification table set; sorting all the verification tables in the verification table set based on the distance between the table to be detected and each verification table, and selecting multiple neighbor verification tables with small distances based on the sorting results.

[0030] Exemplarily, based on the distance between the table to be detected and each neighbor verification table and the predicted label of each neighbor verification table by the network model, the accuracy evaluation parameter of the network model is determined, which may include but is not limited to: using the following formula to determine the accuracy evaluation parameter of the network model: Among them, the value of i ranges from 1 to n, n represents the total number of neighbor verification tables, and d i Indicates the distance between the table to be tested and the i-th nearest neighbor verification table, MAE i It represents the average absolute error between the predicted label of the i-th neighbor verification table and the true label of the i-th neighbor verification table by the network model, P _accuracy Represents the accuracy evaluation parameter.

[0031] Exemplarily, based on the predicted labels of each neighbor verification table by the network model and the predicted labels of each neighbor verification table by other network models other than the network model, the diversity evaluation parameters of the network model are determined, which may include but are not limited to: based on the predicted labels of each neighbor verification table by the network model and the predicted labels of each neighbor verification table by the other network models, a first quantity, a second quantity, a third quantity and a fourth quantity are counted; wherein the first quantity represents the number of neighbor verification tables that are predicted incorrectly by the network model and the other network models, the second quantity represents the number of neighbor verification tables that are predicted incorrectly by the network model and the number of neighbor verification tables that are predicted correctly by the other network models, the third quantity represents the number of neighbor verification tables that are predicted correctly by the network model and the number of neighbor verification tables that are predicted incorrectly by the other network models, and the fourth quantity represents the number of neighbor verification tables that are predicted correctly by the network model and the other network models; wherein, if the difference between the predicted label of the neighbor verification table and the true label of the neighbor verification table is greater than the first threshold and less than the second threshold, the prediction of the neighbor verification table is correct, otherwise, the prediction of the neighbor verification table is incorrect.

[0032] On this basis, based on the first number, the second number, the third number and the fourth number, the similarity parameter between the network model and the other network models is determined; based on the similarity parameter between the network model and each other network model, the diversity evaluation parameter of the network model is determined.

[0033] Exemplarily, if multiple network models are used to implement the classification task, the behavior detection method may also include the following steps: select multiple neighbor verification tables corresponding to the table to be detected of the target object from the verification table set (including multiple verification tables); for each network model, determine the accuracy evaluation parameters of the network model based on the distance between the table to be detected and each neighbor verification table and the predicted label of each neighbor verification table by the network model; select multiple candidate network models from all network models based on the accuracy evaluation parameters of each network model, and determine the target prediction result of the table to be detected based on the multiple candidate network models. The target prediction result is used to detect whether the target object has a specified type of behavior.

[0034] In a possible implementation, in a classification task, based on the distance between the table to be detected and each neighbor verification table and the predicted label of each neighbor verification table by the network model, the accuracy evaluation parameter of the network model is determined, which may include but is not limited to: The accuracy evaluation parameter of the network model may be determined by the following formula: Among them, the value of i ranges from 1 to n, n represents the total number of neighbor verification tables, and d i Indicates the distance between the table to be tested and the i-th nearest neighbor verification table, y i represents the true label of the ith neighbor verification table, y′ iIndicates the probability that the network model predicts the i-th neighbor verification table as the first classification result, 1-y′ i It indicates the probability that the network model predicts the i-th neighbor verification table as the second classification result, P _accuracy Represents the accuracy evaluation parameter.

[0035] Exemplarily, selecting multiple candidate network models from all network models based on the accuracy evaluation parameter and diversity evaluation parameter of each network model may include but is not limited to: sorting all network models based on the accuracy evaluation parameter of each network model, and selecting N1 network models with large accuracy evaluation parameters based on the sorting result; sorting N1 network models based on the diversity evaluation parameters of N1 network models, and selecting M1 network models with large diversity evaluation parameters as candidate network models based on the sorting result. N1 is a positive integer, M1 is a positive integer, and M1 can be less than N1. Alternatively, sorting all network models based on the diversity evaluation parameter of each network model, and selecting N2 network models with large diversity evaluation parameters based on the sorting result; sorting N2 network models based on the accuracy evaluation parameters of N2 network models, and selecting M2 network models with large accuracy evaluation parameters as candidate network models based on the sorting result. N2 is a positive integer, M2 is a positive integer, and M2 can be less than N2.

[0036] Exemplarily, the table to be tested may include table data of K rows and P columns, K is a positive integer, and P is a positive integer. In a possible implementation, the table to be tested includes wind turbine environment data and wind turbine status data collected by the terminal device; the target prediction result is the wind turbine power generation at a future time; the wind turbine power generation is used to predict the wind turbine power generation at a future time to support reasonable wind power allocation.

[0037] In another possible implementation, the table to be tested includes product information (such as product type, product name, product introduction, etc.), product transaction volume, and transaction price collected by the terminal device; the target prediction result is the future product profit. Based on the prediction result of the future product profit, the product can be managed.

[0038] In another possible implementation, the table to be detected includes checkpoint attribute information and checkpoint historical traffic data collected by the terminal device; the target prediction result is the traffic volume passing through the checkpoint in the future; the traffic volume at the checkpoint is used to predict the traffic volume passing through the checkpoint in the future, so as to carry out traffic control on vehicles in a timely manner.

[0039] It can be seen from the above technical scheme that in the embodiment of the present application, multiple neighbor verification tables can be selected from the verification table set, and the accuracy evaluation parameters and diversity evaluation parameters of the network model can be determined based on the predicted labels of each neighbor verification table by the network model. Multiple candidate network models are selected from all network models based on the accuracy evaluation parameters and the diversity evaluation parameters, and the target prediction results of the table to be detected are determined based on the multiple candidate network models, and the target prediction results are used to detect whether the target object has a specified type of behavior. Accurately predict whether the target object has a specified type of behavior in the future time through multiple candidate network models, and then manage the target object to avoid abnormal management of the target object and avoid business failures.

[0040] The behavior detection method of the embodiment of the present application is described below in conjunction with specific embodiments.

[0041] Tabular data is structured data (structured data is also called row data, which is logically expressed and implemented by a two-dimensional table structure and strictly follows data format and length specifications). Tabular data is a common form of data in various industries, such as finance, energy, and medical fields, where machine learning methods are used to solve business problems. When using machine learning methods to solve business problems, tabular data is usually input into a machine learning model, which detects whether the target object has a specified type of behavior, such as predicting whether the target object will have a specified type of behavior in the future.

[0042] However, if the accuracy of the machine learning model is poor, it will not be able to accurately predict whether the target object will have a specified type of behavior in the future, resulting in abnormal management of the target object and business failure.

[0043] For example, if a machine learning model is used to predict the power generation of a wind turbine in the future, and the prediction of the power generation of the wind turbine in the future is inaccurate, the wind turbine cannot be properly deployed, resulting in abnormal management of the wind turbine and even wind turbine failure. That is, when the power generation of the wind turbine in the future is greater than the power threshold, it indicates that a specified type of behavior exists, and wind power deployment is required in this case. However, if the prediction of the power generation of the wind turbine is inaccurate, the detection result of the specified type of behavior will be wrong, which will then lead to abnormal wind power deployment.

[0044] For example, if a machine learning model is used to predict the volume of vehicles passing through a checkpoint in the future, and the prediction of the volume of vehicles passing through the checkpoint is inaccurate, then it is impossible to perform traffic control on vehicles at the checkpoint (such as a highway checkpoint, etc.). For example, if the volume of vehicles passing through the checkpoint in the future is large, if the prediction result is that the volume of vehicles passing through the checkpoint is small, then the flow of vehicles at the checkpoint will not be limited, resulting in excessive volume of vehicles at the checkpoint, which in turn causes congestion at the checkpoint. That is, when the volume of vehicles at a future time is greater than the flow threshold, it indicates that a specified type of behavior exists, and in this case, the flow of vehicles at the checkpoint needs to be limited. However, if the prediction of the volume of vehicles passing through the checkpoint is inaccurate, then the detection result of the specified type of behavior will be wrong, which will in turn lead to the inability to accurately limit the flow of vehicles at the checkpoint, and the flow limiting strategy will be wrong.

[0045] In view of the above findings, the embodiment of the present application proposes a behavior detection method, which selects multiple candidate network models based on the accuracy evaluation parameters and diversity evaluation parameters of the network model, determines the target prediction results of the table to be detected based on the multiple candidate network models, and the target prediction results are used to detect whether the target object has a specified type of behavior. Accurately predict whether the target object has a specified type of behavior in the future through multiple candidate network models, and then manage the target object to avoid abnormal management of the target object.

[0046] See also Figure 2 , which is a flow chart of a behavior detection method, the behavior detection method may include a Generation stage, a Selection stage, and an Aggregation stage.

[0047] In the Generation stage, a batch of network models can be trained using the training set and the validation set and put into the model pool. In the Selection stage, for each test sample in the test set (each test sample is processed separately), a neighboring sample set is selected from the validation set, and the model accuracy and diversity of each network model are evaluated based on the neighboring sample set, and then candidate models (at least two) are selected based on the results of the accuracy and diversity evaluation. In the Aggregation stage, the prediction results of the candidate models on the test samples are weighted and fused according to the model accuracy to obtain the prediction results of multiple candidate models.

[0048] The Generation phase, Selection phase, and Aggregation phase are described below.

[0049] First, the Generation stage.

[0050] In the Generation stage, a training set can be obtained, which may include multiple training tables (i.e., training tables, i.e., sample data in tabular form). Multiple network models can be trained based on the training tables in the training set, and these network models can be deployed on the server, i.e., the server supports these network models.

[0051] Exemplarily, multiple network models may be machine learning models, such as deep learning models or neural network models, etc., without limitation. Multiple network models can implement the same business, such as multiple network models are used to predict the power generation of wind turbines in the future. Or, multiple network models are used to predict whether a user will have overdue credit card payments in the future. Or, multiple network models are used to predict the volume of traffic passing through checkpoints in the future. Of course, the above are just a few examples, and there is no limitation on this business type.

[0052] Exemplarily, any two network models among the multiple network models may be homogeneous network models or heterogeneous network models. A homogeneous network model refers to two network models having the same model structure but different network parameters. For example, both network models are network models of GBDT (Gradient Boosting Decision Tree) structure, but different training parameters are obtained by training the two network models with different training sets.

[0053] Among them, heterogeneous network models refer to: two network models with different model structures, such as RF (Fandom Forests) structure network model, GBDT structure network model, Xgboost (eXtreme Gradient Boosting, distributed gradient enhancement library) structure network model, Gatboost structure network model, LightGBM (Light Gradient Boosting Machine) structure network model, etc.

[0054] Exemplarily, a plurality of network models may be obtained by training with a training set, and different network models may be obtained by training with the same training set or by training with different training sets. During the training process, HPO (HyperParameter Optimization) may be performed, and there is no restriction on this training process.

[0055] Exemplarily, in the Generation stage, a verification set may be obtained, which may include multiple verification tables (ie, verification tables, ie, verification data in a tabular form), each verification table having a true label.

[0056] For each trained network model, the network model can be verified based on the verification set. If the network model passes the verification, the network model is stored in the model pool (the server can obtain the network model from the model pool). If the network model fails the verification, the network model is retrained, and so on, until the network model passes the verification and is stored in the model pool.

[0057] For example, the verification table in the verification set can be input into the network model to obtain the predicted label of the verification table output by the network model. Based on the real label of the verification table and the predicted label of the verification table, the number of successful predictions and the number of failed predictions are counted, and then the prediction success ratio or the prediction failure ratio is determined based on the number of successful predictions and the number of failed predictions, and whether the network model has passed the verification is determined based on the prediction success ratio or the prediction failure ratio. For example, when the prediction success ratio is greater than a threshold, it is determined that the network model has passed the verification, and when the prediction success ratio is not greater than the threshold, it is determined that the network model has not passed the verification.

[0058] In summary, in the Generation phase, multiple network models can be obtained, and these network models can be deployed and stored in the model pool, and the server can obtain these network models from the model pool.

[0059] Second, the Selection stage.

[0060] In the Selection stage, a test set can be obtained, which may include multiple tables to be tested (i.e., tables to be tested, i.e., data to be tested in tabular form). For each table to be tested in the test set, it is necessary to determine the target prediction result of the table to be tested based on the network model. Since the prediction process of each table to be tested is the same, the processing process of the Selection stage is introduced by taking the processing process of a table to be tested as an example.

[0061] Exemplarily, the tasks of the network model can be divided into regression tasks and classification tasks. Among them, the classification task is the process of fitting a model or function that helps to classify data into multiple categories, i.e. discrete values. In the classification task, the data is classified into different labels according to some parameters given in the input. In the classification task, independent features should be used to predict the discrete target variable (category label). In the classification task, a decision boundary needs to be found to separate the different categories in the target variable.

[0062] Regression is the process of finding a model or function that can be used to classify data as continuous real values ​​rather than using categories or discrete values. Regression can identify changes in distribution based on historical data. In regression, independent features should be used to predict continuous target variables.

[0063] If multiple network models (i.e. multiple network models in the model pool) are used to implement regression tasks, see Figure 3 FIG. 4 is a schematic diagram of the processing process of the Selection stage, which may include:

[0064] Step 301: The server selects a plurality of neighbor verification tables corresponding to the to-be-detected table of the target object from a verification table set (ie, verification set). The verification table set may include a plurality of verification tables.

[0065] Exemplarily, the table to be tested in the test set is a table to be tested of the target object, and the table to be tested may include table data of K rows and P columns, where K may be a positive integer and P may be a positive integer.

[0066] For example, the table to be tested includes the fan environment data and fan status data collected by the terminal device, that is, the terminal device collects the fan environment data and fan status data corresponding to the fan (i.e., the target object), and sends the fan environment data and fan status data to the server, and the server stores the fan environment data and fan status data in a table form to obtain the table to be tested. Among them, the fan environment data may include but is not limited to at least one of the following: air pressure data, wind direction data, wind speed data, temperature data, longitude and latitude data, etc. The fan status data may include but is not limited to at least one of the following: fan speed, fan power (such as rated power, current power, etc.). Of course, the above is just an example and there is no limitation to this.

[0067] For example, the table to be tested includes product information (such as product type, product name, product introduction, etc.), product transaction volume, transaction price, etc. collected by the terminal device, that is, the terminal device collects product information, product transaction volume, transaction price, and sends this information to the server, and the server stores this information in a table form to obtain the table to be tested. Of course, the above is just an example and is not limited to this.

[0068] For example, the table to be detected includes the checkpoint attribute information and checkpoint historical traffic data collected by the terminal device, that is, the terminal device collects the checkpoint attribute information and checkpoint historical traffic data corresponding to the checkpoint (such as a high-speed checkpoint, that is, the target object), and sends the checkpoint attribute information and checkpoint historical traffic data to the server. The server stores the checkpoint attribute information and checkpoint historical traffic data in a table form to obtain the table to be detected.

[0069] Of course, the above are just a few examples of the table to be tested, and there is no limitation on the content of the table to be tested.

[0070] Exemplarily, the verification table set may include multiple verification tables, and multiple neighbor verification tables corresponding to the table to be detected may be selected from all verification tables in the verification table set. For example, the distance (such as Euclidean distance, etc.) between the table to be detected and each verification table in the verification table set may be determined, such as extracting the features of the table to be detected and the features of the verification table, and calculating the distance between the table to be detected and the verification table based on the features of the table to be detected and the features of the verification table, and there is no restriction on the distance determination method.

[0071] Based on the distance between the table to be detected and each verification table, all verification tables in the verification table set can be sorted, and multiple neighbor verification tables with small distances can be selected based on the sorting results. For example, all verification tables in the verification table set can be sorted in order of distance from small to large, and starting from the first verification table after sorting, Z verification tables can be selected, and the selected Z verification tables can be used as the neighbor verification tables corresponding to the table to be detected. For another example, all verification tables in the verification table set can be sorted in order of distance from large to small, and starting from the last verification table after sorting, Z verification tables can be selected, and the selected Z verification tables can be used as the neighbor verification tables corresponding to the table to be detected.

[0072] Of course, "distance" is just an example, and other methods can be used to select neighbor verification tables. For example, the similarity between the table to be detected and each verification table in the verification table set can be determined, and based on the similarity between the table to be detected and each verification table, all verification tables in the verification table set can be sorted, and multiple neighbor verification tables with large similarity can be selected based on the sorting results, and there is no limitation on this.

[0073] Step 302: For each network model, the server determines the accuracy evaluation parameters of the network model based on the distance between the table to be detected and each neighbor verification table and the predicted label of each neighbor verification table by the network model, that is, performs a model accuracy evaluation on the network model to obtain the accuracy evaluation parameters.

[0074] Exemplarily, after obtaining multiple neighbor verification tables corresponding to the table to be detected, the distance (such as Euclidean distance) between the table to be detected and each neighbor verification table may also be obtained, and there is no limitation on the acquisition method.

[0075] Exemplarily, after obtaining multiple neighbor verification tables corresponding to the table to be detected, for each network model supported by the server (i.e., each network model in the model pool), the neighbor verification table can be input into the network model, and the network model processes the neighbor verification table to obtain the predicted label of the neighbor verification table, and outputs the predicted label of the neighbor verification table. After inputting multiple neighbor verification tables corresponding to the table to be detected into the network model, the predicted label of each neighbor verification table by the network model can be obtained.

[0076] Exemplarily, based on the distance between the table to be detected and each neighbor verification table, and the predicted label of each neighbor verification table by the network model, the accuracy evaluation parameter of the network model can be determined.

[0077] For example, the following formula (1) can be used to determine the accuracy evaluation parameter of the network model. Of course, the following formula (1) is only an example, and there is no limitation on the method of determining the accuracy evaluation parameter.

[0078]

[0079] In formula (1), the value of i can range from 1 to n, and n can represent the total number of neighbor verification tables. i Indicates the distance between the table to be tested and the i-th nearest neighbor verification table. MAE i It can represent the average absolute error between the predicted label of the i-th neighbor verification table and the true label of the i-th neighbor verification table. For example, based on the predicted label of the i-th neighbor verification table and the true label of the i-th neighbor verification table, the MAE (Mean Absolute Error) formula can be used to calculate the mean absolute error MAE corresponding to the i-th neighbor verification table. i . P_accuracy represents the accuracy evaluation parameter of the network model.

[0080] In summary, based on the distance between the table to be detected and each neighbor verification table and the predicted label of each neighbor verification table by the network model, the accuracy evaluation parameter P_accuracy of the network model can be determined.

[0081] In the above method, the model accuracy in the regression task is evaluated using the MAE method, and the distance between the nearest neighbor verification table and the table to be tested is introduced to optimize the accuracy evaluation of the model for boundary samples.

[0082] Step 303: For each network model, the server determines the diversity evaluation parameters of the network model based on the predicted labels of each neighbor verification table by the network model and the predicted labels of each neighbor verification table by other network models other than the network model (that is, each network model in the remaining network models is regarded as other network models), that is, performs model diversity evaluation on the network model to obtain the diversity evaluation parameters.

[0083] Exemplarily, after obtaining multiple neighbor verification tables corresponding to the table to be detected, for each network model supported by the server (i.e., each network model in the model pool), the neighbor verification table can be input into the network model, and the network model processes the neighbor verification table to obtain the predicted label of the neighbor verification table, and outputs the predicted label of the neighbor verification table. After inputting multiple neighbor verification tables corresponding to the table to be detected into the network model, the predicted label of each neighbor verification table by the network model can be obtained.

[0084] Assuming that the server supports network model a1, network model a2 and network model a3, and the table to be detected corresponds to the neighbor verification table b1 and the neighbor verification table b2, then the predicted label c11 of network model a1 for the neighbor verification table b1, the predicted label c12 of network model a1 for the neighbor verification table b2, the predicted label c21 of network model a2 for the neighbor verification table b1, the predicted label c22 of network model a2 for the neighbor verification table b2, the predicted label c31 of network model a3 for the neighbor verification table b1, and the predicted label c32 of network model a3 for the neighbor verification table b2.

[0085] Exemplarily, for network model a1, the diversity evaluation parameters of network model a1 are determined based on the predicted labels of network model a1 for each neighbor verification table and the predicted labels of other network models (i.e., network model a2 and network model a3) for each neighbor verification table except network model a1.

[0086] Similarly, based on the predicted labels of network model a2 for each neighbor verification table and the predicted labels of other network models (i.e., network model a1 and network model a3) for each neighbor verification table except network model a2, the diversity evaluation parameters of network model a2 are determined. Based on the predicted labels of network model a3 for each neighbor verification table and the predicted labels of other network models (i.e., network model a1 and network model a2) for each neighbor verification table except network model a3, the diversity evaluation parameters of network model a3 are determined.

[0087] Exemplarily, taking the diversity evaluation parameters of network model a1 as an example, based on the predicted labels of each neighbor verification table by network model a1 and the predicted labels of each neighbor verification table by other network models (i.e., network model a2 and network model a3) except network model a1, the diversity evaluation parameters of network model a1 can be determined. For example, the following steps can be used to determine the diversity evaluation parameters of network model a1. Of course, the following method is only an example, and there is no limitation on the method of determining the diversity evaluation parameters.

[0088] Step 3031, based on the predicted labels of each neighbor verification table by the network model (such as network model a1) and the predicted labels of each neighbor verification table by other network models (such as network model a2 or network model a3, taking network model a2 as an example), count the first quantity, the second quantity, the third quantity and the fourth quantity.

[0089] For example, the first number represents the number of errors predicted by the neighbor verification table for both the network model and the other network model, such as the number of errors predicted by the neighbor verification table for both the network model a1 and the network model a2, which is recorded as N 00 For example, based on the predicted labels of the neighbor verification table by network model a1, the prediction error of network model a1 on the neighbor verification table is determined, and based on the predicted labels of the neighbor verification table by network model a2, the prediction error of network model a2 on the neighbor verification table is determined. Then N 00 The value of is added by 1. After the above processing is performed on all neighbor verification tables, the first number N can be obtained. 00 The final value of .

[0090] For example, the second number represents the number of times the network model makes an incorrect prediction of the neighbor verification table and the other network model makes a correct prediction of the neighbor verification table. For example, the number of times the network model a1 makes an incorrect prediction of the neighbor verification table and the network model a2 makes a correct prediction of the neighbor verification table can be recorded as N. 01 For example, based on the predicted labels of the neighbor verification table by network model a1, it is determined that the prediction of the neighbor verification table by network model a1 is wrong, and based on the predicted labels of the neighbor verification table by network model a2, it is determined that the prediction of the neighbor verification table by network model a2 is correct. Then the second number N can be 01 The value of is added by 1. After performing the above processing on all neighbor verification tables, the second number N can be obtained. 01 The final value of .

[0091] For example, the third number represents the number of neighbor verification tables predicted correctly by the network model and the number of neighbor verification tables predicted incorrectly by the other network models. For example, the number of neighbor verification tables predicted correctly by network model a1 and incorrectly predicted by network model a2 can be recorded as N. 10 For example, based on the predicted labels of the neighbor verification table by network model a1, it is determined that the prediction of the neighbor verification table by network model a1 is correct, and based on the predicted labels of the neighbor verification table by network model a2, it is determined that the prediction of the neighbor verification table by network model a2 is wrong, then the third number N can be 10 The value of is added by 1. After the above processing is performed on all neighbor verification tables, the third number N can be obtained. 10 The final value of .

[0092] For example, the fourth number represents the number of correct predictions made by the network model and the other network model according to the neighbor verification table. For example, the number of correct predictions made by the network model a1 and the network model a2 according to the neighbor verification table is recorded as N. 11 For example, based on the predicted labels of the neighbor verification table by network model a1, it is determined that the prediction of network model a1 on the neighbor verification table is correct. Based on the predicted labels of the neighbor verification table by network model a2, it is determined that the prediction of network model a2 on the neighbor verification table is correct. Then N 11 The value of is added by 1. After the above processing is performed on all neighbor verification tables, the fourth number N can be obtained. 11 The final value of .

[0093] Exemplarily, for each neighbor verification table, if the difference between the predicted label of the neighbor verification table and the true label of the neighbor verification table is greater than the first threshold and less than the second threshold, then the prediction of the neighbor verification table is correct, otherwise, the prediction of the neighbor verification table is wrong. For example, if the difference between the predicted label of the neighbor verification table by the network model a1 and the true label of the neighbor verification table is greater than the first threshold and less than the second threshold, then the prediction of the neighbor verification table by the network model a1 is correct, otherwise, the prediction of the neighbor verification table by the network model a1 is wrong. If the difference between the predicted label of the neighbor verification table by the network model a2 and the true label of the neighbor verification table is greater than the first threshold and less than the second threshold, then the prediction of the neighbor verification table by the network model a2 is correct, otherwise, the prediction of the neighbor verification table by the network model a2 is wrong.

[0094] For example, the first threshold and the second threshold may be opposite numbers or may not be opposite numbers. The first threshold and the second threshold are opposite numbers as an example for explanation, as shown in formula (2):

[0095]

[0096] In the above formula, y pred Represents the predicted label of the network model for the neighbor verification table, y true represents the true label of the neighbor verification table, -δ represents the first threshold, and δ represents the second threshold. If N=1, it means that the network model predicts the neighbor verification table correctly, and if N=0, it means that the network model predicts the neighbor verification table incorrectly.

[0097] In formula (2), whether the model prediction is correct is determined by observing whether the predicted value (i.e., predicted label) of the regression task falls within one standard deviation of the true value (i.e., true label), thereby making the prediction result smoother and facilitating the counting of the first quantity, the second quantity, the third quantity, and the fourth quantity.

[0098] Step 3032: Determine a similarity parameter between the network model (such as network model a1) and the other network model (such as network model a2) based on the first quantity, the second quantity, the third quantity and the fourth quantity.

[0099] For example, based on the predicted labels of each neighbor verification table of network model a1 and the predicted labels of each neighbor verification table of network model a2, after counting the first quantity, the second quantity, the third quantity and the fourth quantity, the similarity parameter between network model a1 and network model a2 can be determined based on the first quantity, the second quantity, the third quantity and the fourth quantity. For example, the similarity parameter can be determined using the following formula (3). Of course, the following formula is only an example, and the method for determining the similarity parameter is not limited.

[0100]

[0101] In formula (3), DF i,k N represents the similarity parameter between network model i (i.e., network model a1) and network model k (i.e., network model a2), 00 Represents the first quantity, N 01 Represents the second quantity, N 10 Represents the third quantity, N 11 It represents the fourth quantity, and exp represents the exponential function with the natural constant e as the base.

[0102] Step 3033: Based on the similarity parameters between the network model and each other network model, determine the diversity evaluation parameter of the network model, such as the average value of all similarity parameters as the diversity evaluation parameter.

[0103] For example, based on the similarity parameters between network model a1 and network model a2, and the similarity parameters between network model a1 and network model a3, the diversity evaluation parameters of network model a1 can be determined. For example, these two similarity parameters can be weighted to obtain the diversity evaluation parameters of network model a1.

[0104] Similarly, based on the similarity parameters between the network model a2 and the network model a1, and the similarity parameters between the network model a2 and the network model a3, the diversity evaluation parameters of the network model a2 can be determined.

[0105] Based on the similarity parameters between the network model a3 and the network model a1, and the similarity parameters between the network model a3 and the network model a2, the diversity evaluation parameters of the network model a3 can be determined.

[0106] In summary, based on the predicted labels of each neighbor verification table by the network model and the predicted labels of each neighbor verification table by other network models, the diversity evaluation parameters of the network model can be determined.

[0107] In the above method, the model diversity in the regression task can be evaluated by the similarity parameter using formula (3), which is used to evaluate whether the model is predicting correctly in the regression task. The fewer the prediction errors, the greater the diversity. By calculating the similarities between all models, the diversity value of a single model is obtained.

[0108] Step 304: Based on the accuracy evaluation parameter of each network model and the diversity evaluation parameter of each network model, the server selects multiple network models from all network models as candidate network models.

[0109] In a possible implementation, all network models are sorted based on the accuracy evaluation parameter of each network model, and N1 network models with large accuracy evaluation parameters are selected based on the sorting result, where N1 is a positive integer, such as a positive integer greater than 1. For example, based on the accuracy evaluation parameter of each network model, all network models are sorted in order from large to small according to the accuracy evaluation parameter, and the top N1 network models are selected from the first network model based on the sorting result. Alternatively, based on the accuracy evaluation parameter of each network model, all network models are sorted in order from small to large according to the accuracy evaluation parameter, and the bottom N1 network models are selected from the last network model based on the sorting result.

[0110] After selecting N1 network models from all network models, the N1 network models are sorted based on their diversity evaluation parameters, and M1 network models with large diversity evaluation parameters are selected as candidate network models based on the sorting results, where M1 is a positive integer and may be smaller than N1.

[0111] For example, based on the diversity evaluation parameters of N1 network models, the N1 network models are sorted in descending order according to the diversity evaluation parameters, and the M1 network models with the highest ranking are selected from the first network model based on the sorting results. Alternatively, based on the diversity evaluation parameters of N1 network models, the N1 network models are sorted in descending order according to the diversity evaluation parameters, and the M1 network models with the lowest ranking are selected from the last network model as candidate network models based on the sorting results.

[0112] In a possible implementation, all network models are sorted based on the diversity evaluation parameter of each network model, and N2 network models with large diversity evaluation parameters are selected based on the sorting result, where N2 is a positive integer, such as a positive integer greater than 1. For example, based on the diversity evaluation parameter of each network model, all network models are sorted in descending order of the diversity evaluation parameter, and the top N2 network models are selected from the first network model based on the sorting result. Alternatively, based on the diversity evaluation parameter of each network model, all network models are sorted in descending order of the diversity evaluation parameter, and the bottom N2 network models are selected from the last network model based on the sorting result.

[0113] After selecting N2 network models from all network models, the N2 network models are sorted based on their accuracy evaluation parameters, and M2 network models with large accuracy evaluation parameters are selected as candidate network models based on the sorting results, where M2 is a positive integer and may be less than N2.

[0114] For example, based on the accuracy evaluation parameters of N2 network models, the N2 network models are sorted in descending order according to the accuracy evaluation parameters, and the top M2 network models are selected from the first network model based on the sorting results. Alternatively, based on the accuracy evaluation parameters of N2 network models, the N2 network models are sorted in descending order according to the accuracy evaluation parameters, and the bottom M2 network models are selected from the last network model as candidate network models based on the sorting results.

[0115] Of course, the above is only an example of selecting a candidate network model, and there is no limitation to this. For example, based on the accuracy evaluation parameter and the diversity evaluation parameter of each network model, the accuracy evaluation parameter and the diversity evaluation parameter can be weighted to obtain a weighted score of the network model. Based on the weighted score of each network model, a network model with a large weighted score can be selected as a candidate network model.

[0116] In summary, multiple candidate network models corresponding to the table to be detected can be obtained.

[0117] If multiple network models (i.e. multiple network models in the model pool) are used to implement classification tasks, see Figure 4 FIG. 4 is a schematic diagram of the processing process of the Selection stage, which may include:

[0118] Step 401: The server selects a plurality of neighbor verification tables corresponding to the table to be detected of the target object from a verification table set (ie, verification set). The verification table set may include a plurality of verification tables.

[0119] Exemplarily, the table to be tested in the test set is a table to be tested of the target object, and the table to be tested may include table data of K rows and P columns, where K may be a positive integer and P may be a positive integer.

[0120] The verification table set includes multiple verification tables, and multiple neighbor verification tables corresponding to the table to be detected can be selected from all the verification tables in the verification table set. For example, the distance between the table to be detected and each verification table in the verification table set is determined. Based on the distance between the table to be detected and each verification table, all verification tables in the verification table set are sorted, and multiple neighbor verification tables with small distances are selected based on the sorting results.

[0121] Step 402: For each network model, the server determines the accuracy evaluation parameters of the network model based on the distance between the table to be detected and each neighbor verification table and the predicted label of each neighbor verification table by the network model, that is, performs a model accuracy evaluation on the network model to obtain the accuracy evaluation parameters.

[0122] Exemplarily, after obtaining multiple neighbor verification tables corresponding to the table to be detected, the distance (such as Euclidean distance) between the table to be detected and each neighbor verification table may also be obtained, and there is no limitation on the acquisition method.

[0123] Exemplarily, after obtaining multiple neighbor verification tables corresponding to the table to be detected, for each network model supported by the server (i.e., each network model in the model pool), the neighbor verification table can be input into the network model, and the network model processes the neighbor verification table to obtain the predicted label of the neighbor verification table, and outputs the predicted label of the neighbor verification table. After inputting multiple neighbor verification tables corresponding to the table to be detected into the network model, the predicted label of each neighbor verification table by the network model can be obtained.

[0124] Exemplarily, based on the distance between the table to be detected and each neighbor verification table, and the predicted label of each neighbor verification table by the network model, the accuracy evaluation parameter of the network model can be determined.

[0125] For example, the following formula (4) can be used to determine the accuracy evaluation parameter of the network model. Of course, the following formula (4) is only an example, and there is no limitation on the method of determining the accuracy evaluation parameter.

[0126]

[0127] In formula (4), the value of i can range from 1 to n, and n can represent the total number of neighbor verification tables. i It is used to indicate the distance between the table to be tested and the i-th nearest neighbor verification table. i Used to represent the true label of the i-th neighbor verification table, y′ iUsed to indicate the probability that the network model predicts the i-th neighbor verification table as the first classification result, 1-y i It is used to represent the probability that the network model predicts the i-th neighbor verification table as the second classification result. For example, after the i-th neighbor verification table is input into the network model, the network model can output the probability y that the network model predicts the i-th neighbor verification table as the first classification result. i , and the network model can output the probability 1-y′ that the network model predicts the i-th neighbor verification table as the second classification result i , that is, the sum of the two probabilities is 1. P_accuracy represents the accuracy evaluation parameter of the network model.

[0128] In summary, based on the distance between the table to be detected and each neighbor verification table and the predicted label of each neighbor verification table by the network model, the accuracy evaluation parameter P_accuracy of the network model can be determined.

[0129] In the above method, the Logloss method is used to measure the accuracy of the model to solve the problem of insensitivity when facing boundary samples, because the neighboring sample sets around the boundary samples will have different labels and the proportions may be unbalanced. At the same time, it is proposed to consider the distance between the neighboring samples (i.e., the neighboring validation table) and the test samples (i.e., the table to be tested) when measuring the accuracy of the model, which helps to improve the prediction accuracy of boundary samples.

[0130] Step 403: Based on the accuracy evaluation parameters of each network model, the server selects multiple network models from all network models as candidate network models, namely, multiple candidate network models corresponding to the table to be detected.

[0131] In a possible implementation, all network models are sorted based on the accuracy evaluation parameter of each network model, and M network models with large accuracy evaluation parameters are selected as candidate network models based on the sorting result, where M is a positive integer. For example, based on the accuracy evaluation parameter of each network model, all network models are sorted in order from large to small according to the accuracy evaluation parameter, and based on the sorting result, the top M network models are selected from the first network model as candidate network models. Alternatively, based on the accuracy evaluation parameter of each network model, all network models are sorted in order from small to large according to the accuracy evaluation parameter, and based on the sorting result, the bottom M network models are selected from the last network model as candidate network models. Of course, the above are just examples and are not limited to this.

[0132] In summary, multiple candidate network models corresponding to the table to be detected can be obtained.

[0133] From the above, it can be seen that no matter it is a regression task or a classification task, multiple candidate network models corresponding to the table to be detected can be obtained. At this point, the Selection stage is completed and the Aggregation stage is executed.

[0134] Third, the Aggregation stage.

[0135] For the Aggregation stage, after obtaining multiple candidate network models corresponding to the table to be detected, the server can determine the target prediction result corresponding to the table to be detected based on the multiple candidate network models, and the target prediction result is used to detect whether the target object has a specified type of behavior.

[0136] For example, the table to be tested can be input into a candidate network model, which processes the table to be tested to obtain a prediction result output by the candidate network model. Then, the prediction results output by each candidate network model are weighted and fused to obtain the target prediction result corresponding to the table to be tested.

[0137] Among them, when weighted fusion is performed on the prediction results of each candidate network model, a voting method can be used to weightedly fuse multiple prediction results to obtain a target prediction result, or multiple prediction results can be input into a meta-model to obtain a target prediction result. There is no restriction on this weighted fusion method.

[0138] In one possible implementation, if the table to be tested includes wind turbine environment data and wind turbine status data collected by the terminal device, the target prediction result is the wind turbine power generation at a future time (for example, when the wind turbine power generation is greater than a threshold, it indicates that a specified type of behavior exists, otherwise, there is no specified type of behavior), and the wind turbine power generation is used to predict the wind turbine power generation at a future time to support reasonable wind power allocation.

[0139] For example, a wind power plant needs to give an accurate power generation prediction. This solution can help solve the task of wind turbine power generation prediction. The task of wind turbine power generation prediction is solved using a regression method. After feature engineering, model training and other steps, this solution uses dynamic ensemble learning technology to provide an integrated model with high accuracy and strong generalization to improve the accuracy of the final wind power prediction. In the process of using dynamic ensemble learning technology, the accuracy of multiple models generated during model training is calculated using formula (1), and the diversity of multiple models is calculated using formulas (2) and (3). The topN models are selected according to accuracy, and then the topM models are selected from the topN according to diversity. Finally, the topM models are fused using a weighted method to form a final integrated model with high accuracy and strong generalization.

[0140] In a possible implementation, if the table to be detected includes product information (such as product type, product name, product introduction, etc.), product transaction volume, and transaction price collected by the terminal device, the target prediction result is the future product profit (such as when the product profit is greater than the threshold, it indicates that the specified type of behavior exists, otherwise, the specified type of behavior does not exist). Based on the prediction result of the future product profit, the product can be managed.

[0141] In a possible implementation, the table to be detected includes the checkpoint attribute information and historical traffic data of the checkpoint collected by the terminal device, and the target prediction result is the vehicle flow passing through the checkpoint in the future; the vehicle flow at the checkpoint is used to predict the vehicle flow passing through the checkpoint in the future (such as when the vehicle flow is greater than the threshold, it indicates that a specified type of behavior exists, otherwise, there is no specified type of behavior), so as to perform traffic control on vehicles in a timely manner.

[0142] It can be seen from the above technical solutions that in the embodiments of the present application, multiple neighbor verification tables can be selected from the verification table set, and the accuracy evaluation parameters and diversity evaluation parameters of the network model can be determined based on the prediction labels of each neighbor verification table by the network model. Multiple candidate network models are selected from all network models based on the accuracy evaluation parameters and the diversity evaluation parameters, and the target prediction results of the table to be detected are determined based on the multiple candidate network models, and the target prediction results are used to detect whether the target object has a specified type of behavior in the future. Accurately predict whether the target object will have a specified type of behavior in the future through multiple candidate network models, and then manage the target object to avoid abnormal management of the target object and business failures. Provide prediction results with high accuracy and strong robustness for the table to be detected, provide an instance-level model integration method, select multiple models with high accuracy and strong diversity for the table to be detected, and weightedly fuse the prediction results of the table to be detected by the selected models as the final prediction results, which can ensure that the prediction results are more accurate and robust.

[0143] Based on the same application concept as the above method, a behavior detection device is proposed in the embodiment of the present application, which is applied to a server. The server supports multiple network models. If the multiple network models are used to implement regression tasks, see Figure 5 FIG. 1 is a schematic diagram of the structure of the device, and the device may include:

[0144] A selection module 51 is used to select a plurality of neighbor verification tables corresponding to the table to be detected of the target object from the verification table set; wherein the verification table set includes a plurality of verification tables;

[0145] The determination module 52 is used to determine, for each network model, the accuracy evaluation parameter of the network model based on the distance between the table to be detected and each neighbor verification table and the predicted label of each neighbor verification table by the network model; and determine the diversity evaluation parameter of the network model based on the predicted label of each neighbor verification table by the network model and the predicted label of each neighbor verification table by other network models other than the network model;

[0146] The selection module 51 is further used to select multiple candidate network models from all network models based on the accuracy evaluation parameter of each network model and the diversity evaluation parameter of each network model;

[0147] The determination module 52 is further configured to determine a target prediction result of the table to be detected based on the multiple candidate network models, wherein the target prediction result is used to detect whether the target object has a specified type of behavior.

[0148] Exemplarily, when the selection module 51 selects multiple neighbor verification tables corresponding to the table to be detected of the target object from the verification table set, it is specifically used to: determine the distance between the table to be detected and each verification table in the verification table set; based on the distance between the table to be detected and each verification table, sort all the verification tables in the verification table set, and select multiple neighbor verification tables with small distances based on the sorting results.

[0149] Exemplarily, the determination module 52 determines the accuracy evaluation parameter of the network model based on the distance between the table to be detected and each neighbor verification table and the predicted label of each neighbor verification table by the network model, and is specifically used to determine the accuracy evaluation parameter of the network model using the following formula: Among them, the value of i ranges from 1 to n, n represents the total number of neighbor verification tables, and d i Indicates the distance between the table to be tested and the i-th nearest neighbor verification table, MAE i It represents the average absolute error between the predicted label of the i-th neighbor verification table and the true label of the i-th neighbor verification table by the network model, P _accuracy Represents the accuracy evaluation parameter.

[0150] Exemplarily, the determination module 52 determines the diversity evaluation parameter of the network model based on the predicted label of each neighbor verification table by the network model and the predicted label of each neighbor verification table by other network models other than the network model, and is specifically used to: based on the predicted label of each neighbor verification table by the network model and the predicted label of each neighbor verification table by the other network model, count the first quantity, the second quantity, the third quantity and the fourth quantity; the first quantity represents the number of neighbor verification tables that are predicted incorrectly by the network model and the other network model, the second quantity represents the number of neighbor verification tables that are predicted incorrectly by the network model and the number of neighbor verification tables that are predicted correctly by the other network model, and the third quantity represents the number of neighbor verification tables that are predicted incorrectly by the network model and the number of neighbor verification tables that are predicted correctly by the other network model, and the fourth quantity represents the number of neighbor verification tables that are predicted incorrectly by the network model and the number of neighbor verification tables that are predicted correctly by the other network model. The fourth number represents the number of correct predictions of the neighbor verification table by the network model and the number of incorrect predictions of the neighbor verification table by the other network model, and the fourth number represents the number of correct predictions of the neighbor verification table by the network model and the other network model; wherein, if the difference between the predicted label of the neighbor verification table and the true label of the neighbor verification table is greater than the first threshold and less than the second threshold, the prediction of the neighbor verification table is correct, otherwise, the prediction of the neighbor verification table is wrong; based on the first number, the second number, the third number and the fourth number, determine the similarity parameter between the network model and the other network model; based on the similarity parameter between the network model and each other network model, determine the diversity evaluation parameter of the network model.

[0151] Exemplarily, if the multiple network models are used to implement a classification task, the selection module 51 is also used to select multiple neighbor verification tables corresponding to the table to be detected of the target object from the verification table set; the determination module 52 is also used to determine the accuracy evaluation parameters of the network model for each network model based on the distance between the table to be detected and each neighbor verification table, and the predicted label of the network model for each neighbor verification table; the selection module 51 is also used to select multiple candidate network models from all network models based on the accuracy evaluation parameters of each network model; the determination module 52 is also used to determine the target prediction result of the table to be detected based on the multiple candidate network models; wherein the target prediction result is used to detect whether the target object has a specified type of behavior.

[0152] Exemplarily, the determination module 52 determines the accuracy evaluation parameter of the network model based on the distance between the table to be detected and each neighbor verification table and the predicted label of the network model for each neighbor verification table by: using the following formula to determine the accuracy evaluation parameter of the network model:

[0153]

[0154] Among them, the value of i ranges from 1 to n, n represents the total number of neighbor verification tables, and d iIndicates the distance between the table to be tested and the i-th nearest neighbor verification table, y i represents the true label of the ith neighbor verification table, y′ i Indicates the probability that the network model predicts the i-th neighbor verification table as the first classification result, 1-y′ i It indicates the probability that the network model predicts the i-th neighbor verification table as the second classification result, P _accuracy Represents the accuracy evaluation parameter.

[0155] Exemplarily, when the determination module 52 selects multiple candidate network models from all network models based on the accuracy evaluation parameter and the diversity evaluation parameter of each network model, it is specifically used to:

[0156] All network models are sorted based on the accuracy evaluation parameter of each network model, and N1 network models with large accuracy evaluation parameters are selected based on the sorting results;

[0157] Sorting the N1 network models based on the diversity evaluation parameters of the N1 network models, and selecting M1 network models with large diversity evaluation parameters as candidate network models based on the sorting results;

[0158] Alternatively, all network models are sorted based on the diversity evaluation parameter of each network model, and N2 network models with large diversity evaluation parameters are selected based on the sorting results;

[0159] The N2 network models are sorted based on the accuracy evaluation parameters of the N2 network models, and M2 network models with large accuracy evaluation parameters are selected as candidate network models based on the sorting result.

[0160] Based on the same application concept as the above method, an electronic device (such as a server) is proposed in the embodiment of the present application, see Figure 6 As shown, the server includes: a processor 61 and a machine-readable storage medium 62, the machine-readable storage medium 62 stores machine-executable instructions that can be executed by the processor 61; the processor 61 is used to execute the machine-executable instructions to implement the behavior detection method disclosed in the above example of this application.

[0161] Based on the same application concept as the above method, an embodiment of the present application also provides a machine-readable storage medium, on which a number of computer instructions are stored. When the computer instructions are executed by a processor, the behavior detection method disclosed in the above example of the present application can be implemented.

[0162] The above-mentioned machine-readable storage medium may be any electronic, magnetic, optical or other physical storage device, which may contain or store information, such as executable instructions, data, etc. For example, the machine-readable storage medium may be: RAM (Radom Access Memory), volatile memory, non-volatile memory, flash memory, storage drive (such as hard disk drive), solid state drive, any type of storage disk (such as CD, DVD, etc.), or similar storage medium, or a combination thereof.

[0163] The systems, devices, modules or units described in the above embodiments may be implemented by a computer entity or by a product having a certain function. A typical implementation device is a computer, and the specific form of the computer may be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email transceiver device, a game console, a tablet computer, a wearable device or a combination of any of these devices.

[0164] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0165] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the embodiments of the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0166] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0167] Moreover, these computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0168] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0169] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A behavior detection method, characterized in that: Applied to a server, the server supports multiple network models, if the multiple network models are used to implement a regression task, the method includes: Selecting a plurality of neighbor verification tables corresponding to the to-be-detected table of the target object from the verification table set; For each network model, based on the distance between the table to be detected and each neighbor verification table and the predicted label of each neighbor verification table by the network model, determine the accuracy evaluation parameter of the network model; based on the predicted label of each neighbor verification table by the network model and the predicted label of each neighbor verification table by other network models except the network model, determine the diversity evaluation parameter of the network model; Based on the accuracy evaluation parameter and diversity evaluation parameter of each network model, multiple candidate network models are selected from all network models, and the target prediction result of the table to be detected is determined based on the multiple candidate network models, and the target prediction result is used to detect whether the target object has a specified type of behavior.

2. The method according to claim 1, characterized in that The step of selecting a plurality of neighbor verification tables corresponding to the table to be detected of the target object from the verification table set includes: Determine the distance between the table to be detected and each verification table in the verification table set; Based on the distance between the table to be detected and each verification table, all verification tables in the verification table set are sorted, and a plurality of neighbor verification tables with small distances are selected based on the sorting result.

3. The method according to claim 1, characterized in that The step of determining the accuracy evaluation parameters of the network model based on the distance between the table to be detected and each neighbor verification table and the predicted label of each neighbor verification table by the network model includes: The following formula is used to determine the accuracy evaluation parameters of the network model: Among them, the value of i ranges from 1 to n, n represents the total number of neighbor verification tables, and d i Indicates the distance between the table to be tested and the i-th nearest neighbor verification table, MAE i represents the average absolute error between the predicted label of the i-th neighbor validation table and the true label of the i-th neighbor validation table by the network model, P _accuracy Represents the accuracy evaluation parameter.

4. The method according to claim 1, characterized in that: The method of determining the diversity evaluation parameters of the network model based on the predicted labels of each neighbor verification table by the network model and the predicted labels of each neighbor verification table by other network models other than the network model includes: Based on the predicted labels of each neighbor verification table by the network model and the predicted labels of each neighbor verification table by the other network model, a first quantity, a second quantity, a third quantity and a fourth quantity are counted; wherein, the first quantity represents the number of neighbor verification tables that are predicted incorrectly by the network model and the other network model, the second quantity represents the number of neighbor verification tables that are predicted incorrectly by the network model and the number of neighbor verification tables that are predicted correctly by the other network model, the third quantity represents the number of neighbor verification tables that are predicted correctly by the network model and the number of neighbor verification tables that are predicted incorrectly by the other network model, and the fourth quantity represents the number of neighbor verification tables that are predicted correctly by the network model and the other network model; wherein, if the difference between the predicted label of the neighbor verification table and the true label of the neighbor verification table is greater than the first threshold and less than the second threshold, the prediction of the neighbor verification table is correct, otherwise, the prediction of the neighbor verification table is wrong; Based on the first number, the second number, the third number and the fourth number, a similarity parameter between the network model and the other network models is determined; based on the similarity parameter between the network model and each other network model, a diversity evaluation parameter of the network model is determined.

5. The method according to claim 1, characterized in that If the multiple network models are used to implement a classification task, the method further includes: Selecting a plurality of neighbor verification tables corresponding to the to-be-detected table of the target object from the verification table set; For each network model, based on the distance between the table to be detected and each neighbor verification table and the predicted label of each neighbor verification table by the network model, determine the accuracy evaluation parameter of the network model; A plurality of candidate network models are selected from all network models based on the accuracy evaluation parameters of each network model, and a target prediction result of the table to be detected is determined based on the plurality of candidate network models, wherein the target prediction result is used to detect whether the target object has a specified type of behavior.

6. The method according to claim 5, characterized in that The step of determining the accuracy evaluation parameters of the network model based on the distance between the table to be detected and each neighbor verification table and the predicted label of each neighbor verification table by the network model includes: The following formula is used to determine the accuracy evaluation parameters of the network model: Among them, the value of i ranges from 1 to n, n represents the total number of neighbor verification tables, and d i Indicates the distance between the table to be tested and the i-th nearest neighbor verification table, y i represents the true label of the i-th neighbor verification table, y i ′ represents the probability that the network model predicts the i-th neighbor verification table as the first classification result, 1-y i ′ represents the probability that the network model predicts the i-th neighbor verification table as the second classification result, P _accuracy Represents the accuracy evaluation parameter.

7. The method according to claim 1, characterized in that Based on the accuracy evaluation parameters and diversity evaluation parameters of each network model, multiple candidate network models are selected from all network models, including: All network models are sorted based on the accuracy evaluation parameter of each network model, and N1 network models with large accuracy evaluation parameters are selected based on the sorting results; Sorting the N1 network models based on the diversity evaluation parameters of the N1 network models, and selecting M1 network models with large diversity evaluation parameters as candidate network models based on the sorting results; Alternatively, all network models are sorted based on the diversity evaluation parameter of each network model, and N2 network models with large diversity evaluation parameters are selected based on the sorting results; The N2 network models are sorted based on the accuracy evaluation parameters of the N2 network models, and M2 network models with large accuracy evaluation parameters are selected as candidate network models based on the sorting result.

8. A behavior detection device, characterized in that: Applied to a server, the server supports multiple network models, if the multiple network models are used to implement a regression task, the device comprises: A selection module, used to select a plurality of neighbor verification tables corresponding to the table to be detected of the target object from the verification table set; wherein the verification table set includes a plurality of verification tables; A determination module is used to determine, for each network model, the accuracy evaluation parameters of the network model based on the distance between the table to be detected and each neighbor verification table and the predicted label of each neighbor verification table by the network model; and the diversity evaluation parameters of the network model based on the predicted label of each neighbor verification table by the network model and the predicted label of each neighbor verification table by other network models other than the network model; The selection module is further used to select multiple candidate network models from all network models based on the accuracy evaluation parameter of each network model and the diversity evaluation parameter of each network model; The determination module is further used to determine a target prediction result of the to-be-detected table based on the multiple candidate network models, and the target prediction result is used to detect whether the target object has a specified type of behavior.

9. The device according to claim 8, It is characterized in that in, When the selection module selects multiple neighbor verification tables corresponding to the table to be detected of the target object from the verification table set, it is specifically used to: determine the distance between the table to be detected and each verification table in the verification table set; sort all the verification tables in the verification table set based on the distance between the table to be detected and each verification table, and select multiple neighbor verification tables with small distances based on the sorting result; The determination module is specifically used to determine the accuracy evaluation parameters of the network model based on the distance between the table to be detected and each neighbor verification table and the predicted label of each neighbor verification table by the network model: the accuracy evaluation parameters of the network model are determined by the following formula: Among them, the value of i ranges from 1 to n, n represents the total number of neighbor verification tables, and d i Indicates the distance between the table to be tested and the i-th nearest neighbor verification table, MAE i represents the average absolute error between the predicted label of the i-th neighbor validation table and the true label of the i-th neighbor validation table by the network model, P _accuracy represents the accuracy evaluation parameter; Wherein, the determination module determines the diversity evaluation parameter of the network model based on the predicted label of each neighbor verification table by the network model and the predicted label of each neighbor verification table by other network models except the network model, and is specifically used to: based on the predicted label of each neighbor verification table by the network model and the predicted label of each neighbor verification table by the other network model, count the first quantity, the second quantity, the third quantity and the fourth quantity; the first quantity represents the number of neighbor verification tables that are predicted incorrectly by the network model and the other network model, the second quantity represents the number of neighbor verification tables that are predicted incorrectly by the network model and the number of neighbor verification tables that are predicted correctly by the other network model, and the third quantity represents the number of neighbor verification tables that are predicted incorrectly by the network model and the number of neighbor verification tables that are predicted correctly by the other network model, and the fourth quantity represents the number of neighbor verification tables that are predicted incorrectly by the network model and the number of neighbor verification tables that are predicted correctly by the other network model. The fourth number indicates the number of correct predictions of the neighbor verification table by the network model and the number of incorrect predictions of the neighbor verification table by the other network model, and the fourth number indicates the number of correct predictions of the neighbor verification table by the network model and the other network model; if the difference between the predicted label of the neighbor verification table and the true label of the neighbor verification table is greater than the first threshold and less than the second threshold, the prediction of the neighbor verification table is correct, otherwise, the prediction of the neighbor verification table is wrong; based on the first number, the second number, the third number and the fourth number, determine the similarity parameter between the network model and the other network model; based on the similarity parameter between the network model and each other network model, determine the diversity evaluation parameter of the network model; Wherein, if the multiple network models are used to implement the classification task, the selection module is used to select multiple neighbor verification tables corresponding to the table to be detected of the target object from the verification table set; the determination module is used to determine the accuracy evaluation parameters of the network model for each network model based on the distance between the table to be detected and each neighbor verification table and the predicted label of each neighbor verification table by the network model; the selection module is used to select multiple candidate network models from all network models based on the accuracy evaluation parameters of each network model; the determination module is used to determine the target prediction result of the table to be detected based on the multiple candidate network models, and the target prediction result is used to detect whether the target object has a specified type of behavior; The determination module is specifically used to determine the accuracy evaluation parameters of the network model based on the distance between the table to be detected and each neighbor verification table and the predicted label of each neighbor verification table by the network model: the accuracy evaluation parameters of the network model are determined by the following formula: Among them, the value of i ranges from 1 to n, n represents the total number of neighbor verification tables, and d i Indicates the distance between the table to be tested and the i-th nearest neighbor verification table, y i represents the true label of the i-th neighbor verification table, y i ′ represents the probability that the network model predicts the i-th neighbor verification table as the first classification result, 1-y i ′ represents the probability that the network model predicts the i-th neighbor verification table as the second classification result, P _accuracy represents the accuracy evaluation parameter; Among them, when the determination module 52 selects multiple candidate network models from all network models based on the accuracy evaluation parameters and diversity evaluation parameters of each network model, it is specifically used to: sort all network models based on the accuracy evaluation parameters of each network model, and select N1 network models with large accuracy evaluation parameters based on the sorting results; sort the N1 network models based on the diversity evaluation parameters of the N1 network models, and select M1 network models with large diversity evaluation parameters as candidate network models based on the sorting results; or, sort all network models based on the diversity evaluation parameters of each network model, and select N2 network models with large diversity evaluation parameters based on the sorting results; sort the N2 network models based on the accuracy evaluation parameters of the N2 network models, and select M2 network models with large accuracy evaluation parameters as candidate network models based on the sorting results.

10. An electronic device, characterized in that: include: a processor and a machine-readable storage medium storing machine-executable instructions executable by the processor; The processor is used to execute machine executable instructions to implement the method according to any one of claims 1 to 7.

Images