Data processing model performance improvement method and device, storage medium and electronic device

Abstract

The present invention provides a method and device for improving the performance of a data processing model, a storage medium, and an electronic device. The method includes: reading each data in a determined test data stream one by one, and using the currently read data as target data; obtaining each anomaly detection parameter in a detection module; determining whether concept drift has occurred in the data processing model based on the target data and each anomaly detection parameter; if so, triggering the detection module to update each anomaly detection parameter and retraining the data processing model; after the data processing model training is completed, using the next read data as target data, obtaining each updated anomaly detection parameter in the detection module, and returning to execute the operation of determining whether concept drift has occurred in the data processing model until all data in the test data stream are read. When concept drift occurs in the data processing model, the data processing model is retrained, thereby improving the performance of the data processing model.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method and device for improving the performance of a data processing model, a storage medium, and an electronic device. Background Art

[0002] With the development of neural networks, more and more industries use neural networks to build data processing models, and then use the data processing models to process the data, thereby realizing functions such as data classification and prediction.

[0003] Data processing models are typically trained successfully using known data before processing new, unknown data. However, in many scenarios dominated by dynamic streaming data, such as finance and transportation, the distribution of the data stream and the information it reflects change over time and in different environments. This phenomenon of shifting data distribution or information over time is known as concept drift, and concept drift can reduce the accuracy of data processing models and lower their performance. Summary of the Invention

[0004] In view of this, the present invention provides a method and device for improving the performance of a data processing model, a storage medium, and an electronic device. After detecting that concept drift exists in a data processing model, the data processing model is retrained to reduce the impact of concept drift on the data processing model and improve the performance of the data processing model.

[0005] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:

[0006] The first aspect of the present application discloses a method for improving the performance of a data processing model, comprising:

[0007] Determine a test data stream of a data processing model, read each data in the test data stream one by one, and use the currently read data as target data;

[0008] Obtain various anomaly detection parameters pre-generated in the detection module;

[0009] Performing concept drift detection on the data processing model based on the target data and each of the anomaly detection parameters;

[0010] When it is determined that concept drift occurs in the data processing model, triggering the detection module to update various anomaly detection parameters and retrain the data processing model;

[0011] After the data processing model training is completed, the data read next time is used as the target data, and the updated anomaly detection parameters in the detection module are obtained, and the operation of performing concept drift detection on the data processing model based on the target data and the anomaly detection parameters is returned to execute until all the data in the test data stream are read.

[0012] In the above method, optionally, the process of the detection module generating various anomaly detection parameters includes:

[0013] Determining a training data set corresponding to the data processing model, and determining a detection data set within the training data set;

[0014] Processing the detection data set to obtain upper edge parameters and lower edge parameters of the detection data set;

[0015] Based on the upper edge parameter and the lower edge parameter, obtaining a minimum anomaly ratio parameter of the detection data set;

[0016] The upper edge parameter, the lower edge parameter, and the minimum anomaly ratio parameter are all determined as anomaly detection parameters.

[0017] In the above method, optionally, performing concept drift detection on the data processing model based on the target data and each of the anomaly detection parameters includes:

[0018] Saving the target data into a received data set, and obtaining statistics on the number of data in the received data set;

[0019] Determining a total number of abnormal data in the received data set based on the upper edge parameter and the lower edge parameter in each of the abnormality detection parameters;

[0020] Based on the minimum anomaly ratio parameter in each of the anomaly detection parameters, the number statistics, and the number summary, whether concept drift occurs in the data processing model is detected.

[0021] In the above method, optionally, performing concept drift detection on the data processing model based on the target data and each of the anomaly detection parameters includes:

[0022] Determining an upper edge parameter and a lower edge parameter in each of the abnormality detection parameters, and determining the upper edge parameter, the lower edge parameter, and a preset drift confidence as drift operation parameters;

[0023] Storing the target data in a detection set, and segmenting each data in the detection set to obtain a first detection interval, a second detection interval, a third detection interval, and a fourth detection interval;

[0024] Calling a preset interval statistical algorithm to process the first detection interval, the second detection interval, the third detection interval, and the fourth detection interval respectively to obtain a statistical parameter group for the first detection interval, a statistical parameter group for the second detection interval, a statistical parameter group for the third detection interval, and a statistical parameter group for the fourth detection interval;

[0025] Processing the statistical parameter group of the first detection interval, the statistical parameter group of the second detection interval, and each of the drift operation parameters to obtain a first drift boundary value and a first drift detection value;

[0026] Processing the statistical parameter group of the third detection interval, the statistical parameter group of the fourth detection interval, and each of the drift operation parameters to obtain a second drift boundary value and a second drift detection value;

[0027] Based on the first drift boundary value, the first drift detection value, the second drift boundary value and the second drift detection value, it is detected whether concept drift occurs in the data processing model.

[0028] The above method may optionally further include, after performing concept drift detection on the data processing model:

[0029] Determining an upper edge parameter and a lower edge parameter in each of the abnormality detection parameters, and determining the upper edge parameter, the lower edge parameter, and a preset warning confidence as warning operation parameters;

[0030] Processing the statistical parameter group of the first detection interval, the statistical parameter group of the second detection interval, and each of the early warning operation parameters to obtain a first early warning boundary value and a first early warning detection value;

[0031] Processing the statistical parameter group of the third detection interval, the statistical parameter group of the fourth detection interval, and each of the early warning operation parameters to obtain a second early warning boundary value and a second early warning detection value;

[0032] Based on the first warning boundary value, the first warning detection value, the second warning boundary value and the second warning detection value, it is determined whether the data processing model meets the warning conditions, and when the data processing model meets the warning conditions, a warning message is sent.

[0033] Optionally, the method described above includes segmenting each data in the detection set to obtain a first detection interval, a second detection interval, a third detection interval, and a fourth detection interval, including:

[0034] Determine first segmentation data and second segmentation data in the detection set;

[0035] determining each data in the detection set that is located before the first segmented data and the first segmented data as a first detection interval;

[0036] determining each data in the detection set that is located after the first segmented data and the first segmented data as a second detection interval;

[0037] determining each data in the detection set that is located before the second segmented data and the second segmented data as a third detection interval;

[0038] Each data in the detection set that is located after the second segmented data and the second segmented data are determined as a fourth detection interval.

[0039] Optionally, the above method further includes, after obtaining the first detection interval, the second detection interval, the third detection interval, and the fourth detection interval:

[0040] Calling the interval statistical algorithm to process the detection set to obtain a statistical parameter group of the detection set;

[0041] Acquire a first segmentation boundary based on the statistical parameter group of the detection set and each of the drift operation parameters;

[0042] acquiring a second segmentation boundary based on the statistical parameter group of the first detection interval and each of the drift operation parameters;

[0043] acquiring a third segmentation boundary based on the statistical parameter group of the third detection interval and each of the drift operation parameters;

[0044] determining whether the first segmented data needs to be updated based on the first segmentation boundary, the second segmentation boundary, the statistical parameter group of the detection set, and the statistical parameter group of the first detection interval; and updating the target data to the first segmented data in the detection set if it is determined that the first segmented data needs to be updated;

[0045] Based on the first segmentation boundary, the third segmentation boundary, the statistical parameter group of the detection set, and the statistical parameter group of the third detection interval, determine whether the second segmentation data needs to be updated; when it is determined that the second segmentation data needs to be updated, update the target data to the second segmentation data in the detection set.

[0046] The second aspect of the present application discloses a device for improving the performance of a data processing model, comprising:

[0047] a reading unit, configured to determine a test data stream of a data processing model, read each data in the test data stream one by one, and use the currently read data as target data;

[0048] A first acquisition unit is used to acquire various anomaly detection parameters pre-generated in the detection module;

[0049] a detection unit, configured to perform concept drift detection on the data processing model based on the target data and each of the anomaly detection parameters;

[0050] a triggering unit, configured to trigger the detection module to update various anomaly detection parameters and retrain the data processing model when determining that concept drift occurs in the data processing model;

[0051] The return execution unit is used to use the data read next time as the target data after the data processing model training is completed, and obtain the updated anomaly detection parameters in the detection module, and return to execute the operation of performing concept drift detection on the data processing model based on the target data and the various anomaly detection parameters until all data in the test data stream are read.

[0052] The above device may optionally further include:

[0053] a first determining unit, configured to determine a training data set corresponding to the data processing model, and to determine a detection data set within the training data set;

[0054] a first processing unit, configured to process the detection data set to obtain an upper edge parameter and a lower edge parameter of the detection data set;

[0055] an obtaining unit, configured to obtain a minimum anomaly ratio parameter of the detection data set based on the upper edge parameter and the lower edge parameter;

[0056] The second determining unit is configured to determine the upper edge parameter, the lower edge parameter, and the minimum anomaly ratio parameter as anomaly detection parameters.

[0057] In the above device, optionally, the detection unit includes:

[0058] A saving subunit, configured to save the target data into a received data set and obtain statistics on the number of data in the received data set;

[0059] A first determining subunit is configured to determine a total number of abnormal data in the received data set based on an upper edge parameter and a lower edge parameter in each of the abnormality detection parameters;

[0060] The first detection subunit is configured to detect whether concept drift occurs in the data processing model based on a minimum anomaly ratio parameter in each of the anomaly detection parameters, the number statistics, and the number summary.

[0061] In the above device, optionally, the detection unit includes:

[0062] a second determining subunit, configured to determine an upper edge parameter and a lower edge parameter from each of the abnormality detection parameters, and determine the upper edge parameter, the lower edge parameter, and a preset drift confidence as drift operation parameters;

[0063] a segmentation subunit, configured to store the target data into a detection set, and segment each data in the detection set to obtain a first detection interval, a second detection interval, a third detection interval, and a fourth detection interval;

[0064] a calling subunit, configured to call a preset interval statistical algorithm to process the first detection interval, the second detection interval, the third detection interval, and the fourth detection interval, respectively, to obtain a statistical parameter group for the first detection interval, a statistical parameter group for the second detection interval, a statistical parameter group for the third detection interval, and a statistical parameter group for the fourth detection interval;

[0065] a first processing subunit, configured to process the statistical parameter group of the first detection interval, the statistical parameter group of the second detection interval, and each of the drift operation parameters to obtain a first drift boundary value and a first drift detection value;

[0066] a second processing subunit, configured to process the statistical parameter group of the third detection interval, the statistical parameter group of the fourth detection interval, and each of the drift operation parameters to obtain a second drift boundary value and a second drift detection value;

[0067] The second detection subunit is configured to detect whether concept drift occurs in the data processing model based on the first drift boundary value, the first drift detection value, the second drift boundary value, and the second drift detection value.

[0068] The above device may optionally further include:

[0069] a third determining unit, configured to determine an upper edge parameter and a lower edge parameter from each of the abnormality detection parameters, and determine the upper edge parameter, the lower edge parameter, and a preset warning confidence as warning operation parameters;

[0070] a second processing unit, configured to process the statistical parameter group of the first detection interval, the statistical parameter group of the second detection interval, and each of the early warning operation parameters to obtain a first early warning boundary value and a first early warning detection value;

[0071] a third processing unit, configured to process the statistical parameter group of the third detection interval, the statistical parameter group of the fourth detection interval, and each of the early warning operation parameters to obtain a second early warning boundary value and a second early warning detection value;

[0072] The fourth determination unit is used to determine whether the data processing model meets the warning conditions based on the first warning boundary value, the first warning detection value, the second warning boundary value and the second warning detection value, and send warning information when the data processing model meets the warning conditions.

[0073] In the above device, optionally, the cutting subunit comprises:

[0074] a first determining module, configured to determine each data in the detection set that is located before the first segmented data and the first segmented data as a first detection interval;

[0075] a second determining module, configured to determine each data in the detection set located after the first segmented data and the first segmented data as a second detection interval;

[0076] a third determining module, configured to determine each data in the detection set that is located before the second segmented data and the second segmented data as a third detection interval;

[0077] The fourth determining module is configured to determine each data in the detection set that is located after the second segmented data and the second segmented data as a fourth detection interval.

[0078] The above device may optionally further include:

[0079] a calling unit, configured to call the interval statistical algorithm to process the detection set to obtain a statistical parameter group of the detection set;

[0080] A second acquiring unit, configured to acquire a first segmentation boundary based on the statistical parameter group of the detection set and each of the drift operation parameters;

[0081] a third acquiring unit, configured to acquire a second segmentation boundary based on the statistical parameter group of the first detection interval and each of the drift operation parameters;

[0082] a fourth acquiring unit, configured to acquire a third segmentation boundary based on the statistical parameter group of the third detection interval and each of the drift operation parameters;

[0083] a first determining unit, configured to determine whether the first segmented data needs to be updated based on the first segmentation boundary, the second segmentation boundary, the statistical parameter group of the detection set, and the statistical parameter group of the first detection interval, and update the target data to the first segmented data in the detection set when it is determined that the first segmented data needs to be updated;

[0084] A second judgment unit is used to judge whether the second segmentation data needs to be updated based on the first segmentation boundary, the third segmentation boundary, the statistical parameter group of the detection set, and the statistical parameter group of the third detection interval, and when it is determined that the second segmentation data needs to be updated, the target data is updated to the second segmentation data in the detection set.

[0085] A third aspect of the present application discloses a storage medium, which includes stored instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to execute the data processing model performance improvement method as described above.

[0086] The fourth aspect of the present application discloses an electronic device comprising a memory and one or more instructions, wherein the one or more instructions are stored in the memory and configured to be executed by one or more processors as described above in the method for improving data processing model performance.

[0087] Compared with the prior art, the present invention has the following advantages:

[0088] The present invention provides a method and device for improving the performance of a data processing model, a storage medium, and an electronic device. The method includes: determining a test data stream of the data processing model, reading each data in the test data stream one by one, and using the currently read data as target data; obtaining each anomaly detection parameter in a detection module; performing concept drift detection on the data processing model based on the target data and each anomaly detection parameter; when it is determined that concept drift has occurred in the data processing model, triggering the detection module to update each anomaly detection parameter and retraining the data processing model; after the data processing model training is completed, using the next read data as the target data, obtaining each updated anomaly detection parameter in the detection module, and returning to perform concept drift detection on the data processing model based on the target data and each anomaly detection parameter until all data in the test data stream are read. By applying the present invention, when concept drift is detected in the data processing model, the data processing model is retrained, thereby reducing the impact of concept drift on the accuracy of data processing by the data processing model, thereby improving the performance of the data processing model. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0090] Figure 1 A method flow chart of a method for improving data processing model performance provided by an embodiment of the present invention;

[0091] Figure 2 A flow chart of a method for generating various anomaly detection parameters by a detection module provided in an embodiment of the present invention;

[0092] Figure 3 A flow chart of a method for detecting concept drift in a data processing model provided by an embodiment of the present invention;

[0093] Figure 4 A flow chart of another method for detecting concept drift in a data processing model provided by an embodiment of the present invention;

[0094] Figure 5 A flow chart of a method for determining whether to send an early warning message provided by an embodiment of the present invention;

[0095] Figure 6 A flowchart of updating segmented data provided by an embodiment of the present invention;

[0096] Figure 7 A flowchart of another method for improving the performance of a data processing model provided by an embodiment of the present invention;

[0097] FIG8( a ) is a schematic diagram of an application case provided by an embodiment of the present invention;

[0098] FIG8( b ) is another application schematic diagram of an application case provided by an embodiment of the present invention;

[0099] FIG8( c ) is another application schematic diagram of an application case provided by an embodiment of the present invention;

[0100] Figure 9 A schematic structural diagram of a device for improving data processing model performance provided by an embodiment of the present invention;

[0101] Figure 10 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0102] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0103] In this application, the terms "comprises," "comprising," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0104] The present invention can be applied to a wide variety of general-purpose or specialized computing device environments or configurations. For example, personal computers, server computers, handheld or portable devices, tablet devices, multi-processor devices, and distributed computing environments including any of the above. The methods provided in embodiments of the present invention can be applied to model checking systems for improving model performance, where the systems are comprised of various computer terminals or intelligent service devices, and the execution entities can be processors or servers.

[0105] Reference Figure 1 , which is a flow chart of a method for improving the performance of a data processing model provided by an embodiment of the present invention, is specifically described as follows:

[0106] S101 , determining a test data stream of a data processing model, reading each data in the test data stream one by one, and using the currently read data as target data.

[0107] A test data stream is data that requires processing by a data processing model. It contains multiple data sets, each of which is arranged in an orderly queue. Data processing models can be any data processing model, such as AI models or models for data classification.

[0108] After determining the test data stream of the data processing model, each data in the test data stream is acquired one by one, and the currently acquired data is used as the target data.

[0109] S102: Acquire various anomaly detection parameters pre-generated in the detection module.

[0110] The detection model is used to generate various anomaly detection parameters. This is based on the training dataset of the data processing model and a preset outlier detection method. Specifically, the outlier detection method may be the Tukey box plot method. Specifically, the anomaly detection parameters include an upper edge parameter, a lower edge parameter, and a minimum anomaly ratio parameter. Optionally, the detection module may also be referred to as an upper and lower bound estimation module.

[0111] S103: Based on the target data and various anomaly detection parameters, perform concept drift detection on the data processing model.

[0112] When performing concept drift detection on a data processing model based on target data and various anomaly detection parameters, concept drift detection can be performed on the data processing model from two perspectives: the Hoffding boundary or the data anomaly ratio.

[0113] Preferably, the concept drift detection module may use a preset concept drift detection algorithm to process the target data and various anomaly detection parameters, thereby implementing concept drift detection on the data processing model.

[0114] S104: When it is determined that concept drift occurs in the data processing model, the detection module is triggered to update various anomaly detection parameters and retrain the data processing model.

[0115] When the detection module updates each anomaly detection parameter, the detection module generates each anomaly detection parameter based on the process of generating anomaly detection parameters, and uses the generated each anomaly detection parameter as the updated each anomaly detection parameter, that is, the generated each anomaly detection parameter is the each anomaly detection parameter pre-generated in the detection module.

[0116] When retraining the data processing model, the training data set of the data processing model is updated first. Specifically, each data obtained from the test data stream is updated as a training sample in the training data set, and the oldest training sample in the training data set is removed to complete the update of the training data set. The data processing model is then retrained using the updated training data set.

[0117] It should be noted that the capacity of the data pool used to store the training dataset is fixed. To reduce storage costs, when new training samples are added to the training dataset, old training samples need to be deleted, thereby reducing data storage costs. By retraining the data processing model with the updated training dataset, the data processing model can be trained promptly when concept drift occurs, reducing the impact of concept drift on the data processing model's subsequent data processing, thereby improving the data processing model's performance.

[0118] Preferably, when it is determined that no concept drift has occurred in the data processing model, the data read next time is used as the target data, and the process returns to S102 until all the data in the test data stream are read; when it is determined that no concept drift has occurred in the data processing model, the data processing model can be triggered to process the target data; for example, assuming that the data processing model is used to classify data, the data processing module is triggered to classify the target data and output the data type of the target data.

[0119] Optionally, considering the simplicity of system execution, when the various anomaly detection data in the detection model have not been updated, the various anomaly detection data of the previous target data can be directly used as the various anomaly detection data of the current target detection data. Therefore, when it is determined that no concept drift has occurred in the data processing model, it is possible to return to execution S103.

[0120] S105. After the data processing model training is completed, the data read next time is used as the target data, and the updated anomaly detection parameters in the detection module are obtained. The operation of performing concept drift detection on the data processing model based on the target data and the various anomaly detection parameters is returned until all the data in the test data stream are read.

[0121] Optionally, after the data processing model training is completed, the trained data processing model can be used to process the target data, and then the operation of using the data read next time as the target data is performed.

[0122] In the method provided by an embodiment of the present invention, a test data stream of a data processing model is determined, each data in the test data stream is read one by one, and the currently read data is used as the target data; each anomaly detection parameter in the detection module is obtained; based on the target data and each anomaly detection parameter, concept drift detection is performed on the data processing model; when it is determined that concept drift has occurred in the data processing model, the detection module is triggered to update each anomaly detection parameter and the data processing model is retrained; after the data processing model training is completed, the next data read is used as the target data, and the updated anomaly detection parameters in the detection module are obtained, and the operation of performing concept drift detection on the data processing model based on the target data and each anomaly detection parameter is returned until all data in the test data stream are read. By applying the present invention, when concept drift is detected in the data processing model, the data processing model is retrained, thereby reducing the impact of concept drift on the data processing model and improving the performance of the data processing model.

[0123] In another embodiment provided by the present invention, referring to Figure 2 , which is the process of generating various anomaly detection parameters for the detection module. The details are as follows:

[0124] S201: Determine a training data set corresponding to a data processing model, and determine a detection data set in the training data set.

[0125] The training dataset is a collection of data samples used to train the data processing model. The training dataset contains a large number of training samples, which are also known as data. The test dataset is determined within the training dataset. The training samples within the training dataset can be updated. When concept drift is detected in the data processing model, the training dataset is updated. The test dataset can then be determined within the updated training samples to obtain anomaly parameters that better match the current state of the data processing model.

[0126] When determining the test data set in the training data set, N data are selected from the training data set, and the selected N data are used to form the test data set. It should be noted that when selecting data, data with a storage time closer to the current time is selected to form the test data set.

[0127] S202: Process the detection data set to obtain upper edge parameters and lower edge parameters of the detection data set.

[0128] The detection data set is processed using an outlier detection method, wherein the outlier detection method can be specifically a Tukey box plot method. The detection data set is processed using the Tukey box plot method to obtain the first quantile and the second quantile, wherein the first quantile can also be called the 1 / 4 quantile, represented by Q1; the second quantile can also be called the 3 / 4 quantile, represented by Q3.

[0129] The first quartile and the second quartile are calculated to obtain the interquartile range. Specifically, the interquartile range IQR = Q3-Q1. IQR covers the middle 50% of the data distribution in the test data set and is robust.

[0130] The third quartile and the interquartile range are calculated using the preset upper edge calculation formula to obtain the upper edge parameter; the first quartile and the interquartile range are calculated using the preset lower edge calculation formula to obtain the lower edge parameter;

[0131] It should be noted that the upper edge calculation formula is: b = Q3 + 1.5 * IQR;

[0132] The specific formula for calculating the lower edge is: a=Q1-1.5*IQR;

[0133] The parameters in the upper edge calculation formula and the lower edge calculation formula are explained: b represents the upper edge parameter; a represents the lower edge parameter; Q1 represents the first quantile; Q3 represents the second quantile; and IQR represents the interquartile range.

[0134] S203 : Based on the upper edge parameter and the lower edge parameter, obtain a minimum anomaly ratio parameter of the detection data set.

[0135] Determine the total number of abnormal data in the detection data set based on the upper edge parameter and the lower edge parameter; wherein the abnormal data is data in the detection data set that is less than the lower edge parameter, or data in the detection data set that is greater than the upper edge parameter;

[0136] The ratio of the total number of abnormal data to the total number of data in the test data set is used as the minimum anomaly ratio parameter, specifically: θ = outlier(d) / length(d); where θ represents the minimum anomaly ratio parameter; outlier(d) represents the total number of abnormal data in the test data set; and length(d) represents the total number of data in the test data set.

[0137] S204: Determine the upper edge parameter, the lower edge parameter, and the minimum anomaly ratio parameter as anomaly detection parameters.

[0138] By using the outlier detection method to process the detection data set, various anomaly detection parameters can be extracted from the training data set of the data processing model, so that it is possible to more effectively determine whether the data processing model has concept drift. The training data set can be updated, and then the anomaly detection parameters can be updated, so that it can adapt to a variety of application scenarios. In addition, the data processing model can update the anomaly detection parameters in a timely manner during the data processing process, making the concept drift detection of the data processing model more accurate.

[0139] In another embodiment provided by the present invention, referring to Figure 3 , is a flow chart of a method for detecting concept drift in a data processing model based on target data and various anomaly detection parameters, where, Figure 3 The detection method shown is based on the abnormal ratio of data. Figure 3 The instructions are as follows:

[0140] S301: Save target data into a received data set, and obtain statistics on the number of data in the received data set.

[0141] The received data set stores the data read from the test data stream. After the target data is stored in the received data set, the number of data in the received data set is counted. When counting the number of data in the received data set, a counter can be used. After the target data is stored in the received data set, the counter is incremented by one, thereby obtaining the number of additional data in the received data set after the target data is stored in the received data set.

[0142] The specific statistics of the number of data in the received data set is: sum=sum+1(init sum=0), where init sum=0 indicates that the initial value of sum is 0.

[0143] S302: Determine the total number of abnormal data in the received data set based on the upper edge parameter and the lower edge parameter in each of the abnormality detection parameters.

[0144] When determining the total number of abnormal data in the received data set, a number counter can be used for statistics. The number counter is used to count the number of abnormal data in the received data set. When the target data is abnormal data, the counter is incremented by one to obtain the total number of abnormal data in the received data set. When the target data is not abnormal data, the counter is not incremented by one, and the current value of the counter is used as the total number of abnormal data in the received data set. Specifically, as follows: counter = counter + 1 (init counter = 0), where init counter = 0 indicates that the initial value of counter is 0.

[0145] When judging whether the target data is abnormal data, it is determined whether the target data is less than the lower edge parameter or greater than the upper edge parameter; if the target data is less than the lower edge parameter or greater than the upper edge parameter, the target data is determined to be abnormal data.

[0146] Furthermore, abnormal data refers to data in the received data set that is smaller than a lower edge parameter or larger than an upper edge parameter.

[0147] S303: Detect whether concept drift occurs in the data processing model based on the minimum anomaly ratio parameter in each of the anomaly detection parameters, the number statistics, and the number summary.

[0148] When detecting whether a data processing model has undergone concept drift, the ratio of the number statistic and the number summary is determined to determine whether the ratio is greater than the product of the minimum anomaly ratio parameter and a preset constant; if so, it is determined that the data processing model has undergone concept drift; if not, it is not determined that the data processing model has undergone concept drift.

[0149] Specifically, the ratio of the number statistics to the number summary is: The preset constant is 2, and the minimum anomaly ratio parameter is expressed as θ; then When the concept drift occurs in the data processing model, it is determined.

[0150] In another embodiment provided by the present invention, referring to Figure 4, is a flow chart of another method for concept drift detection of data processing models based on target data and various anomaly detection parameters, where, Figure 4 The detection method shown is to detect from the Hoefding boundary. Figure 4 The instructions are as follows:

[0151] S401 : Determine an upper edge parameter and a lower edge parameter in each abnormality detection parameter, and determine the upper edge parameter, the lower edge parameter, and a preset drift confidence as drift operation parameters.

[0152] It should be noted that the lower edge parameter is represented by a; the upper edge parameter is represented by b, and the drift confidence level is represented by α, where α = 0.001;

[0153] S402: Store the target data into a detection set, and segment each data in the detection set to obtain a first detection interval, a second detection interval, a third detection interval, and a fourth detection interval.

[0154] The detection set is used to store data read from the test data stream. The process of segmenting each data in the detection set to obtain the first detection interval, the second detection interval, the third detection interval, and the fourth detection interval is as follows:

[0155] Determine first segmentation data and second segmentation data in the detection set;

[0156] Determine each data before the first segmented data in the detection set and the first segmented data as a first detection interval;

[0157] Determine each data after the first segmented data in the detection set and the first segmented data as a second detection interval;

[0158] Determine each data before the second segmented data in the detection set and the second segmented data as a third detection interval;

[0159] Each data after the second segmented data in the detection set and the second segmented data are determined as a fourth detection interval.

[0160] Two segmentation data are set in the detection set. The two segmentation data are the data in the detection set, and the detection set is segmented using the two segmentation data.

[0161] Detection set using X 总 Indicates that X 总 ={x1,x2,x3,......x n}; where x i Indicates the data read from the test data stream, which can be accessed using x iRepresents the target data. After storing the target data in the detection set, the x in the detection set is i That is x n .

[0162] The first slice of data uses x incre_cut Indicates that the second split data uses x decre_cut Indicates that, the initial x incre_cut =x1, initial x decre_cut = x1; The first segmentation data and the second segmentation data provided in the embodiment of the present invention are dynamically changing data. The specific update process can be referred to the following for Figure 5 Description of the process.

[0163] Specifically, the first detection interval is X incre_cut ={x1,x2,x3,...,x incre_cut}; The second detection interval is X n-incre_cut ={x incre_cut ,...,x n}; The third detection interval is X decre_cut ={x1,x2,x3,...,x decre_cut}; The fourth detection interval is X n-decre_cut ={x decre_cut ,...,x n}.

[0164] S403. Call a preset interval statistical algorithm to process the first detection interval, the second detection interval, the third detection interval, and the fourth detection interval respectively to obtain a statistical parameter group for the first detection interval, a statistical parameter group for the second detection interval, a statistical parameter group for the third detection interval, and a statistical parameter group for the fourth detection interval.

[0165] The statistical parameter group consists of the exponentially weighted moving average value EWAM(X) and the step value ε(X). The interval statistical algorithm consists of the calculation method of the exponentially weighted moving average value EWAM and the calculation method of the step value ε. The calculation process of the exponentially weighted moving average value EWAM is shown in the following formula:

[0166]

[0167]

[0168] Wherein, λ is the preset weight attenuation coefficient, λ=0.05; when using the interval statistics algorithm to calculate the first detection interval X incre_cut ={x1,x2,x3,...,x incre_cut}, the second detection interval X n-incre_cut ={x incre_cut ,...,x n}, the third detection interval X decre_cut ={x1,x2,x3,...,x decre_cut} and the fourth detection interval X n-decre_cut ={x decre_cut ,...,x n}When processing, the first data of each detection interval is recorded as x1.

[0169] The statistical parameter group of the first detection interval includes EWEA(X incre_cut ) and ε(X incre_cut ); The statistical parameter group of the second detection interval includes EWEA(X n-incre_cut ) and ε(X n-incre_cut ); The statistical parameter group of the third detection interval includes EWEA(X decre_cut ) and ε(X decre_cut ); The statistical parameter group of the fourth detection interval includes EWEA(X n-decre_cut ) and ε(X n-decre_cut ).

[0170] S404 : Process the statistical parameter group of the first detection interval, the statistical parameter group of the second detection interval, and each drift operation parameter to obtain a first drift boundary value and a first drift detection value.

[0171] The exponentially weighted moving average value EWEA(X n-incre_cut ) and the exponentially weighted moving average EWEA(X incre_cut ) performs a subtraction operation to obtain a first drift detection value; the first drift detection value is specifically: EWEA(X n-incre_cut )-EWEA(X incre_cut );

[0172] The step value ε(X incre_cut ), the step value ε(X n-incre_cut ), the lower edge parameter a, the upper edge parameter b and the drift confidence α are substituted into the preset drift boundary value calculation formula to perform calculation, thereby obtaining the first drift boundary value drift_bound(X incre ), the calculation process is as follows:

[0173]

[0174] S405 : Process the statistical parameter group of the third detection interval, the statistical parameter group of the fourth detection interval, and each drift operation parameter to obtain a second drift boundary value and a second drift detection value.

[0175] The exponentially weighted moving average value EWEA(X n-decre_cut) and the exponentially weighted moving average EWEA(X decre_cut ) is subtracted to obtain a second drift detection value; the second drift detection value is specifically: EWEA(X decre_cut )-EWEA(X n-decre_cut );

[0176] The step value ε(X decre_cut ), the step value ε(X n-decre_cut ), the lower edge parameter a, the upper edge parameter b and the drift confidence α are substituted into the preset drift boundary value calculation formula to perform calculation, thereby obtaining the second drift boundary value drift_bound (X decre ), the calculation process is as follows:

[0177]

[0178] S406: Detect whether concept drift occurs in the data processing model based on the first drift boundary value, the first drift detection value, the second drift boundary value, and the second drift detection value.

[0179] In the process of detecting whether concept drift occurs in the data processing model, when the first drift detection value is greater than the first drift boundary value, or the second drift detection value is greater than the second drift boundary value, it is determined that concept drift occurs in the data processing model. It should be noted that when the first drift detection value is greater than the first drift boundary value, it is determined that the type of concept drift that occurs in the data processing model is increase type drift; when the second drift detection value is greater than the first drift boundary value, it is determined that the type of concept drift that occurs in the data processing model is decrease type drift; when the first drift detection value is not greater than the first drift boundary value and the second drift detection value is not greater than the second drift boundary value, it is determined that concept drift does not occur in the data processing model.

[0180] The present invention divides the detection set into multiple detection intervals, calculates the statistical parameter group of each detection interval, processes the statistical parameter group of each detection interval and each drift operation parameter, obtains a first drift boundary value, a first drift detection value, a second drift boundary value and a second drift detection value, and detects whether concept drift occurs in the data processing model based on the first drift boundary value, the first drift detection value, the second drift boundary value and the second drift detection value. Furthermore, when it is detected that concept drift occurs in the data processing model, the type of concept drift that occurs in the data processing model can be further determined, which can provide a training basis for subsequent retraining of the data processing model, thereby improving the performance of the data processing model.

[0181] continue Figure 4Continuing to explain, by segmenting the detection set and processing the intervals obtained after segmentation, concept drift detection is performed on the data processing model. In another embodiment provided by the present invention, in addition to performing concept drift detection on the data processing model, it is also possible to determine whether it is necessary to send an early warning message. The specific process is referred to Figure 5 , the specific instructions are as follows:

[0182] S501 , determining an upper edge parameter and a lower edge parameter in each abnormality detection parameter, and determining the upper edge parameter, the lower edge parameter and a preset warning confidence as warning operation parameters.

[0183] The warning confidence level warning_level is represented by β, where β = 0.005.

[0184] S502: Process the statistical parameter group of the first detection interval, the statistical parameter group of the second detection interval, and each early warning operation parameter to obtain a first early warning boundary value and a first early warning detection value.

[0185] The exponentially weighted moving average value EWEA(X n-incre_cut ) and the exponentially weighted moving average EWEA(X incre_cut ) is subtracted to obtain the first warning detection value; the first warning detection value is specifically: EWEA(X n-incre_cut )-EWEA(X incre_cut );

[0186] The step value ε(X incre_cut ), the step value ε(X n-incre_cut ), the lower edge parameter a, the upper edge parameter b and the warning confidence β are substituted into the preset warning boundary value calculation formula to calculate, thereby obtaining the first warning boundary value warning_bound(X incre ), the calculation process is as follows:

[0187]

[0188] S503: Process the statistical parameter group of the third detection interval, the statistical parameter group of the fourth detection interval, and each warning operation parameter to obtain a second warning boundary value and a second warning detection value.

[0189] The exponentially weighted moving average value EWEA(X n-decre_cut ) and the exponentially weighted moving average EWEA(X decre_cut ) is subtracted to obtain the second warning detection value; the second warning detection value is specifically: EWEA(X decre_cut )-EWEA(Xn-decre_cut );

[0190] The step value ε(X decre_cut ), the step value ε(X n-decre_cut ), the lower edge parameter a, the upper edge parameter b and the warning confidence are substituted into the preset warning boundary value calculation formula to calculate the second drift boundary value warning_bound(X decre ), the calculation process is as follows:

[0191]

[0192] S504. Determine whether the data processing model meets the warning condition based on the first warning boundary value, the first warning detection value, the second warning boundary value, and the second warning detection value, and send warning information when the data processing model meets the warning condition.

[0193] When determining whether the data processing model meets the warning conditions, the first warning boundary value and the first warning detection value are compared, and the second warning boundary value and the second warning detection value are compared. When the first warning detection value is greater than the first warning boundary value, or the second warning detection value is greater than the second warning boundary value, it is determined that the data processing model meets the warning conditions; when the first warning detection value is not greater than the first warning boundary value, and the second warning detection value is not greater than the second warning boundary value, it is determined that the data processing model does not meet the warning conditions.

[0194] When the warning information is text information, the warning information includes the results of judgment based on the first warning boundary value, the first warning detection value, the second warning boundary value and the second warning detection value. The warning information can be sent to the staff to complete the warning.

[0195] By determining whether an early warning is needed, early warning information can be issued in a timely manner to remind staff of the current status of the data processing model.

[0196] continue Figure 4 The present invention provides another embodiment for Figure 4 The first segmentation data x mentioned in incre_cut and the second split data x decre_cut For the update, refer to Figure 6 , the specific instructions are as follows:

[0197] S601: Call an interval statistical algorithm to process the detection set to obtain a statistical parameter group of the detection set.

[0198] The interval statistics algorithm can be referred to above Figure 4 The instructions in will not be repeated here.

[0199] Detection set X 总 The statistical parameter group includes the exponentially weighted moving average EWAM (X 总 ) and the step value ε(X 总 ).

[0200] S602: Acquire a first segmentation boundary based on the statistical parameter group of the detection set and each of the drift operation parameters.

[0201] The step value of the detection set and each drift operation parameter are processed using a preset segmentation boundary operation method to obtain a first segmentation boundary.

[0202] The specific method of dividing the boundary is as follows: Where bound(X) represents the segmentation boundary, a represents the lower edge parameter, b represents the upper edge parameter, ε(X) represents the step value, and α represents the drift confidence.

[0203] Therefore, the first segmentation boundary

[0204] S603 : Acquire a second segmentation boundary based on the statistical parameter group of the first detection interval and each of the drift operation parameters.

[0205] The step value of the first detection interval and each drift operation parameter are processed using the segmentation boundary operation method to obtain a second segmentation boundary. For details about the segmentation boundary operation, please refer to S502 and will not be repeated here. The second segmentation boundary

[0206] S604 : Acquire a third segmentation boundary based on the statistical parameter group of the third detection interval and each of the drift operation parameters.

[0207] The step value of the third detection interval and each drift operation parameter are processed using the segmentation boundary operation method to obtain the third segmentation boundary. For the description of the segmentation boundary operation, please refer to S502 and will not be repeated here. The third segmentation boundary

[0208] S605. Based on the first segmentation boundary, the second segmentation boundary, the statistical parameter group of the detection set, and the statistical parameter group of the first detection interval, determine whether the first segmentation data needs to be updated. When it is determined that the first segmentation data needs to be updated, update the target data to the first segmentation data in the detection set.

[0209] Determine the EWAM(X 总 ) and the first segmentation boundary bound(X 总 ) is less than or equal to the EWEA(X incre_cut) and the second segmentation boundary bound(X incre_cut ) between the sum; if not, it is determined that the first segmentation data does not need to be updated, and the first segmentation data remains unchanged; if so, it is determined that the first segmentation data needs to be updated, and the target data is updated to the first segmentation data in the detection combination, that is, x incre_cut =x i , where x i is the target data.

[0210] Furthermore, in EWAM(X 总 )+bound(X 总 )≤EWEA(X incre_cut )+bound(X incre_cut )hour;

[0211] Then EWEA(X incre_cut )=EWAM(X 总 );

[0212] bound(X incre_cut )=bound(X 总 );

[0213] x incre_cut =x i ;

[0214] The above is the process of updating the first segmented data.

[0215] S606. Based on the first segmentation boundary, the third segmentation boundary, the statistical parameter group of the detection set, and the statistical parameter group of the third detection interval, determine whether the second segmentation data needs to be updated. When it is determined that the second segmentation data needs to be updated, update the target data to the second segmentation data in the detection set.

[0216] Determine the EWAM(X 总 ) and the first segmentation boundary bound(X 总 ) is less than or equal to the EWEA(X decre_cut ) and the second segmentation boundary bound(X decre_cut ) between the difference; if not, it is determined that the second segmentation data does not need to be updated, and the second segmentation data remains unchanged; if so, it is determined that the second segmentation data needs to be updated, and the target data is updated to the second segmentation data in the detection combination, that is, x decre_cut =x i , where x i is the target data.

[0217] Furthermore, in EWAM(X 总 )-bound(X总 )≤EWEA(X incre_cut )-bound(X incre_cut )hour;

[0218] Then EWEA(X decre_cut )=EWAM(X 总 );

[0219] bound(X decre_cut )=bound(X 总 );

[0220] x decre_cut =x i ;

[0221] The above is the process of updating the second segmentation data.

[0222] Reference Figure 7 , which is a flow chart of another method for improving the performance of a data processing model provided by an embodiment of the present invention. Figure 7 The specific instructions are as follows:

[0223] The training data is stored in a historical data pool, which is used to store historical data, both labeled and unlabeled. Labeled data, such as model training data, is used to initialize the upper and lower bound parameters of the detection algorithm. Data collected from online data streams is generally unlabeled. This provides persistent storage for online data and serves as a basis for parameter initialization when restarting the detection algorithm after drift. The size of the historical data pool is fixed, storing only the latest buffer size of data. Older data is removed from the pool. This is because as data continues to grow, storage costs will continue to increase. Furthermore, a certain amount of data closest to the current time is sufficient to express the characteristics and distribution of the current time and environment.

[0224] The upper and lower bound estimation module processes data from the historical data pool and outputs upper and lower edge parameters, as well as the minimum anomaly ratio parameter. This upper and lower bound estimation module is similar to the detection module described above. The latest n data points (initially, all training data) are collected from the data pool to estimate the upper and lower bounds of the data under the current data distribution and output the proportion of anomaly data.

[0225] The concept drift detection module includes drift detection algorithms, including but not limited to the Hoffding bound detection algorithm and the anomaly ratio detection algorithm. The concept drift detection algorithm determines whether drift has occurred based on data acquired from the online data stream and the upper and lower bound parameters, lower and minimum anomaly ratio parameters, from the upper and lower bound estimation modules. If drift does not occur, the algorithm continues to determine whether drift has occurred based on data acquired from the online data stream until all data from the online data stream is available. If drift occurs, the data processing module is retrained, or an alert is issued, causing the upper and lower bound estimation modules to recalculate the upper and lower bound parameters, lower and minimum anomaly ratio parameters, and continue to determine whether drift has occurred based on the recalculated upper and lower bound parameters, minimum anomaly ratio parameters, and data acquired from the online data stream until all data from the online data stream is available. The concept drift detection module: In response to the increasing amount of online streaming data, the algorithm detects concept drift using the Hoffding bound and anomaly ratio metrics. The Hoffding bound detection algorithm is the same as HDDM-W, with confidence levels of 0.001 and 0.005 used as the Hoffding bound thresholds for calculating the drift and warning levels, respectively, to determine the current drift status. The data anomaly ratio is calculated from the algorithm startup. Based on the upper and lower bound parameters determined by the upper and lower bound modules, the proportion of abnormal data in the statistics is calculated. If it exceeds a certain threshold, the data distribution is considered to be strongly inconsistent, indicating concept drift. If drift is detected, appropriate drift warnings are issued, and the data processing model is retrained. Simultaneously, the upper and lower bounds are re-estimated based on the updated data pool, and relevant variables within the detection algorithm are updated. The drift detection module is reset based on the new upper and lower bounds. This new drift detection module then continuously monitors subsequent online data streams.

[0226] The present invention breaks away from the limitation of applying the judgment condition based on the Hoeffing boundary to the unlabeled input data (real-time labels are required). An approximate estimation method, the Tukey box plot method, is used for the upper and lower bounds of the input data to meet the application conditions of the Hoeffing inequality and the McGrady inequality. In the process of using the Hoeffing boundary for drift detection, the influence of abnormal data points beyond the upper and lower bounds is taken into account, and the proportion of abnormal data is used as one of the conditions for the occurrence of drift. The present invention breaks away from the requirements of the traditional DDM algorithm for data labels, and does not need to artificially introduce additional parameter values ​​in data selection like the detection method based on statistical features, while taking into account the influence of abnormal data on concept drift.

[0227] Here we use an application case to illustrate the following:

[0228] The original dataset consists of 20 hours of electrocardiograms downloaded from Physionet. 5,000 heartbeats were randomly selected. The task is to classify the heartbeats of patients with severe congestive heart failure. The input data has a dimension of 140, the number of categories is 5, the training dataset size is 500, and the test dataset size is 4,500.

[0229] (1) The model uses a multi-layer perceptron network (two-layer fully connected MLP). First, it is trained on the training set to obtain a trained model. The accuracy of the model on the training set is 99.40%. At the same time, it is tested on the test set, and the overall accuracy of the test set is 92.20%. It can be seen that due to concept drift, the model effect on the test set has a significant decline. In order to observe the accuracy change in a more fine-grained manner, the 4500 test data are divided into 45 data blocks (time periods) of 100 pieces each according to the time series, and tested. The accuracy results are as follows:

[0230] Accuracy:[1.0,0.99,1.0,0.99,1.0,1.0,1.0,1.0,0.97,1.0,0.99,0.99,0.99,1.0,1.0,0.99,1.0,0.98,1.0,1.0,0.99,0.99,0. 99,1.0,0.99,1.0,0.88,0.81,0.8,0.74,0.87,0.87,0.86,0.58,0.61,0.76,0.64,0.85,0.96,0.9,0.97,0.96,0.92,0.85,0.81];

[0231] The line graph of the accuracy results is shown in Figure 8(a). As shown in Figure 8(a), concept drift begins to occur after the 2500th data point, and subsequently has varying degrees of impact on the accuracy of the model.

[0232] (2) In order to detect drift without labels (and thus unable to calculate accuracy), we use our detection method based on the Hoeffding bounds test.

[0233] First, for each dimension of the training data, the upper and lower bound estimation modules are used to output the upper edge parameter a, the lower edge parameter b, and the minimum anomaly ratio parameter θ.

[0234] Secondly, the test data of each dimension is input into the concept drift detection module as a data stream, and combined with the upper and lower bound parameters calculated above, it is determined whether drift occurs.

[0235] Finally, the detection results for each data dimension are integrated. That is, if drift is detected in any dimension, it is considered that concept drift has occurred at this data point. The final drift detection results are output.

[0236] As shown in Figure 8(b), the detected drift data points are roughly consistent with the performance degradation of the model (accuracy), which can timely and effectively detect and warn concept drift in the absence of labels.

[0237] (3) Based on the calculation results of concept drift, the model is retrained (fine-tuned) and its improvement effect is observed. The specific settings are as follows: the 4500 test data are divided into 45 data blocks (time periods) of 100 items in time series, and predictions are made for each time period. At the same time, based on the detection results of the algorithm, it is determined which of the 45 time periods have concept drift. If concept drift exists in a certain time period t, the model is fine-tuned using the data of time period t-1 at time t, and the new model is used for subsequent predictions.

[0238] As shown in Figure 8(c), which is a line graph of the model's accuracy on data processed after retraining at the drift point, after fine-tuning the model based on our concept drift calculation results, the overall accuracy significantly improved compared to the original model. The curve "accuracy" represents the accuracy of the original model, and the curve "accuracy_retrain" represents the accuracy of the fine-tuned model. The average accuracy of the curve "accuracy_retrain" reached 94.00%, and the average accuracy of the curve "accuracy" reached 92.20%. For the average accuracy of the 26th to 45th time period (after concept drift began), the curve "accuracy" reached 82.94%, and the curve "accuracy_retrain" reached 86.63%.

[0239] After detecting concept drift, the model is retrained to improve its performance.

[0240] and Figure 1 Correspondingly, the present invention also provides a data processing model performance improvement device for supporting Figure 1 The method shown in the figure is implemented in real life. The device is used in a computer terminal. The structural diagram of the device is shown in FIG. Figure 9 The specific instructions are as follows:

[0241] The reading unit 801 is used to determine the test data stream of the data processing model, read each data in the test data stream one by one, and use the currently read data as the target data;

[0242] A first acquisition unit 802 is used to acquire various anomaly detection parameters pre-generated in the detection module;

[0243] A detection unit 803 is configured to perform concept drift detection on the data processing model based on the target data and each of the anomaly detection parameters;

[0244] a triggering unit 804 configured to trigger the detection module to update various anomaly detection parameters and retrain the data processing model when determining that concept drift occurs in the data processing model;

[0245] Return to the execution unit 805, which is used to use the data read next time as the target data after the data processing model training is completed, and obtain the updated anomaly detection parameters in the detection module, and return to execute the operation of performing concept drift detection on the data processing model based on the target data and the various anomaly detection parameters until all data in the test data stream are read.

[0246] In the device provided by the embodiment of the present invention, a test data stream of a data processing model is determined, each data in the test data stream is read one by one, and the currently read data is used as the target data; each anomaly detection parameter in the detection module is obtained; based on the target data and each anomaly detection parameter, concept drift detection is performed on the data processing model; when it is determined that concept drift has occurred in the data processing model, the detection module is triggered to update each anomaly detection parameter and retrain the data processing model; after the data processing model training is completed, the next data read is used as the target data, and the updated anomaly detection parameters in the detection module are obtained, and the operation of performing concept drift detection on the data processing model based on the target data and each anomaly detection parameter is returned until all data in the test data stream are read. By applying the present invention, when concept drift is detected in the data processing model, the data processing model is retrained, thereby reducing the impact of concept drift on the data processing model and improving the performance of the data processing model.

[0247] The device provided in the embodiment of the present invention may also be configured as follows:

[0248] a first determining unit, configured to determine a training data set corresponding to the data processing model, and to determine a detection data set within the training data set;

[0249] a first processing unit, configured to process the detection data set to obtain an upper edge parameter and a lower edge parameter of the detection data set;

[0250] an obtaining unit, configured to obtain a minimum anomaly ratio parameter of the detection data set based on the upper edge parameter and the lower edge parameter;

[0251] The second determining unit is configured to determine the upper edge parameter, the lower edge parameter, and the minimum anomaly ratio parameter as anomaly detection parameters.

[0252] In the device provided by the embodiment of the present invention, the detection unit 803 may be configured as follows:

[0253] A saving subunit, configured to save the target data into a received data set and obtain statistics on the number of data in the received data set;

[0254] A first determining subunit is configured to determine a total number of abnormal data in the received data set based on an upper edge parameter and a lower edge parameter in each of the abnormality detection parameters;

[0255] The first detection subunit is configured to detect whether concept drift occurs in the data processing model based on a minimum anomaly ratio parameter in each of the anomaly detection parameters, the number statistics, and the number summary.

[0256] In the device provided by the embodiment of the present invention, the detection unit 803 may be configured as follows:

[0257] a second determining subunit, configured to determine an upper edge parameter and a lower edge parameter from each of the abnormality detection parameters, and determine the upper edge parameter, the lower edge parameter, and a preset drift confidence as drift operation parameters;

[0258] a segmentation subunit, configured to store the target data into a detection set, and segment each data in the detection set to obtain a first detection interval, a second detection interval, a third detection interval, and a fourth detection interval;

[0259] a calling subunit, configured to call a preset interval statistical algorithm to process the first detection interval, the second detection interval, the third detection interval, and the fourth detection interval, respectively, to obtain a statistical parameter group for the first detection interval, a statistical parameter group for the second detection interval, a statistical parameter group for the third detection interval, and a statistical parameter group for the fourth detection interval;

[0260] a first processing subunit, configured to process the statistical parameter group of the first detection interval, the statistical parameter group of the second detection interval, and each of the drift operation parameters to obtain a first drift boundary value and a first drift detection value;

[0261] a second processing subunit, configured to process the statistical parameter group of the third detection interval, the statistical parameter group of the fourth detection interval, and each of the drift operation parameters to obtain a second drift boundary value and a second drift detection value;

[0262] The second detection subunit is configured to detect whether concept drift occurs in the data processing model based on the first drift boundary value, the first drift detection value, the second drift boundary value, and the second drift detection value.

[0263] The device provided in the embodiment of the present invention may also be configured as follows:

[0264] a third determining unit, configured to determine an upper edge parameter and a lower edge parameter from each of the abnormality detection parameters, and determine the upper edge parameter, the lower edge parameter, and a preset warning confidence as warning operation parameters;

[0265] a second processing unit, configured to process the statistical parameter group of the first detection interval, the statistical parameter group of the second detection interval, and each of the early warning operation parameters to obtain a first early warning boundary value and a first early warning detection value;

[0266] a third processing unit, configured to process the statistical parameter group of the third detection interval, the statistical parameter group of the fourth detection interval, and each of the early warning operation parameters to obtain a second early warning boundary value and a second early warning detection value;

[0267] The fourth determination unit is used to determine whether the data processing model meets the warning conditions based on the first warning boundary value, the first warning detection value, the second warning boundary value and the second warning detection value, and send warning information when the data processing model meets the warning conditions.

[0268] In the device provided by the embodiment of the present invention, the cutting subunit may be configured as follows:

[0269] a first determining module, configured to determine each data in the detection set that is located before the first segmented data and the first segmented data as a first detection interval;

[0270] a second determining module, configured to determine each data in the detection set located after the first segmented data and the first segmented data as a second detection interval;

[0271] a third determining module, configured to determine each data in the detection set that is located before the second segmented data and the second segmented data as a third detection interval;

[0272] The fourth determining module is configured to determine each data in the detection set that is located after the second segmented data and the second segmented data as a fourth detection interval.

[0273] The device provided in the embodiment of the present invention may also be configured as follows:

[0274] a calling unit, configured to call the interval statistical algorithm to process the detection set to obtain a statistical parameter group of the detection set;

[0275] A second acquiring unit, configured to acquire a first segmentation boundary based on the statistical parameter group of the detection set and each of the drift operation parameters;

[0276] a third acquiring unit, configured to acquire a second segmentation boundary based on the statistical parameter group of the first detection interval and each of the drift operation parameters;

[0277] a fourth acquiring unit, configured to acquire a third segmentation boundary based on the statistical parameter group of the third detection interval and each of the drift operation parameters;

[0278] a first determining unit, configured to determine whether the first segmented data needs to be updated based on the first segmentation boundary, the second segmentation boundary, the statistical parameter group of the detection set, and the statistical parameter group of the first detection interval, and update the target data to the first segmented data in the detection set when it is determined that the first segmented data needs to be updated;

[0279] A second judgment unit is used to judge whether the second segmentation data needs to be updated based on the first segmentation boundary, the third segmentation boundary, the statistical parameter group of the detection set, and the statistical parameter group of the third detection interval, and when it is determined that the second segmentation data needs to be updated, the target data is updated to the second segmentation data in the detection set.

[0280] An embodiment of the present invention further provides a storage medium, wherein the storage medium includes stored instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to perform the following operations:

[0281] Determine a test data stream of a data processing model, read each data in the test data stream one by one, and use the currently read data as target data;

[0282] Obtain various anomaly detection parameters pre-generated in the detection module;

[0283] Performing concept drift detection on the data processing model based on the target data and each of the anomaly detection parameters;

[0284] When it is determined that concept drift occurs in the data processing model, triggering the detection module to update various anomaly detection parameters and retrain the data processing model;

[0285] After the data processing model training is completed, the data read next time is used as the target data, and the updated anomaly detection parameters in the detection module are obtained, and the operation of performing concept drift detection on the data processing model based on the target data and the anomaly detection parameters is returned to execute until all the data in the test data stream are read.

[0286] The embodiment of the present invention further provides an electronic device, the structural diagram of which is shown in FIG. Figure 10 As shown, the system specifically includes a memory 901 and one or more instructions 902, wherein the one or more instructions 902 are stored in the memory 901 and are configured to be executed by one or more processors 903 to perform the following operations:

[0287] Determine a test data stream of a data processing model, read each data in the test data stream one by one, and use the currently read data as target data;

[0288] Obtain various anomaly detection parameters pre-generated in the detection module;

[0289] Performing concept drift detection on the data processing model based on the target data and each of the anomaly detection parameters;

[0290] When it is determined that concept drift occurs in the data processing model, triggering the detection module to update various anomaly detection parameters and retrain the data processing model;

[0291] After the data processing model training is completed, the data read next time is used as the target data, and the updated anomaly detection parameters in the detection module are obtained, and the operation of performing concept drift detection on the data processing model based on the target data and the anomaly detection parameters is returned to execute until all the data in the test data stream are read.

[0292] The specific implementation processes and derivative methods of the above embodiments are all within the protection scope of the present invention.

[0293] Each embodiment in this specification is described in a progressive manner. The same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple. For relevant parts, refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without expending creative work.

[0294] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0295] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for improving the performance of a data processing model, characterized in that: include: Determine a test data stream of a data processing model, read each data in the test data stream one by one, and use the currently read data as target data; The data in the test data stream is electrocardiogram data; the data processing model is used to classify the electrocardiogram data; Acquire various anomaly detection parameters pre-generated in the detection module; each of the anomaly detection parameters includes an upper edge parameter, a lower edge parameter, and a minimum anomaly ratio parameter; Performing concept drift detection on the data processing model based on the target data and each of the anomaly detection parameters; When it is determined that concept drift occurs in the data processing model, triggering the detection module to update various anomaly detection parameters and retrain the data processing model; After the data processing model training is completed, the data read next time is used as the target data, and the updated anomaly detection parameters in the detection module are obtained, and the operation of performing concept drift detection on the data processing model based on the target data and the anomaly detection parameters is returned to execute until all data in the test data stream are read; Using the retrained data processing model to classify the electrocardiogram data, and obtaining the classification results of the electrocardiogram data; Determine an upper edge parameter and a lower edge parameter in each anomaly detection parameter, and determine the upper edge parameter, the lower edge parameter and a preset warning confidence as warning operation parameters; Processing the statistical parameter group of the first detection interval, the statistical parameter group of the second detection interval, and each early warning operation parameter to obtain a first early warning boundary value and a first early warning detection value; Processing the statistical parameter group of the third detection interval, the statistical parameter group of the fourth detection interval, and each early warning operation parameter to obtain a second early warning boundary value and a second early warning detection value; When the first warning detection value is greater than the first warning boundary value, and / or the second warning detection value is greater than the second warning boundary value, determining that the data processing model meets the warning condition; When the first warning detection value is not greater than the first warning boundary value, and the second warning detection value is not greater than the second warning boundary value, it is determined that the data processing model does not meet the warning conditions; when the data processing model meets the warning conditions, a warning message is sent; the first detection interval, the second detection interval, the third detection interval and the fourth detection interval are obtained by segmenting the data in the detection set; the detection set is used to save the data read from the test data stream; the statistical parameter group of the first detection interval, the second detection interval, the third detection interval and the fourth detection interval is obtained by using a preset interval statistical algorithm to process the first detection interval, the second detection interval, the third detection interval and the fourth detection interval respectively, and the statistical parameter group includes an exponentially weighted moving average and a step value.

2. The method according to claim 1, characterized in that The process by which the detection module generates various anomaly detection parameters includes: Determining a training data set corresponding to the data processing model, and determining a detection data set within the training data set; Processing the detection data set to obtain upper edge parameters and lower edge parameters of the detection data set; Based on the upper edge parameter and the lower edge parameter, obtaining a minimum anomaly ratio parameter of the detection data set; The upper edge parameter, the lower edge parameter, and the minimum anomaly ratio parameter are all determined as anomaly detection parameters.

3. The method according to claim 1, characterized in that The performing concept drift detection on the data processing model based on the target data and each of the anomaly detection parameters includes: Saving the target data into a received data set, and obtaining statistics on the number of data in the received data set; Determining a total number of abnormal data in the received data set based on the upper edge parameter and the lower edge parameter in each of the abnormality detection parameters; Based on the minimum anomaly ratio parameter in each of the anomaly detection parameters, the number statistics, and the number summary, whether concept drift occurs in the data processing model is detected.

4. The method according to claim 1, wherein The performing concept drift detection on the data processing model based on the target data and each of the anomaly detection parameters includes: Determining an upper edge parameter and a lower edge parameter in each of the abnormality detection parameters, and determining the upper edge parameter, the lower edge parameter, and a preset drift confidence as drift operation parameters; Storing the target data in a detection set, and segmenting each data in the detection set to obtain a first detection interval, a second detection interval, a third detection interval, and a fourth detection interval; Calling a preset interval statistical algorithm to process the first detection interval, the second detection interval, the third detection interval, and the fourth detection interval respectively to obtain a statistical parameter group for the first detection interval, a statistical parameter group for the second detection interval, a statistical parameter group for the third detection interval, and a statistical parameter group for the fourth detection interval; Processing the statistical parameter group of the first detection interval, the statistical parameter group of the second detection interval, and each of the drift operation parameters to obtain a first drift boundary value and a first drift detection value; Processing the statistical parameter group of the third detection interval, the statistical parameter group of the fourth detection interval, and each of the drift operation parameters to obtain a second drift boundary value and a second drift detection value; Based on the first drift boundary value, the first drift detection value, the second drift boundary value and the second drift detection value, it is detected whether concept drift occurs in the data processing model.

5. The method according to claim 4, characterized in that The segmenting of each data in the detection set to obtain a first detection interval, a second detection interval, a third detection interval, and a fourth detection interval includes: Determine first segmentation data and second segmentation data in the detection set; determining each data in the detection set that is located before the first segmented data and the first segmented data as a first detection interval; determining each data in the detection set that is located after the first segmented data and the first segmented data as a second detection interval; determining each data in the detection set that is located before the second segmented data and the second segmented data as a third detection interval; Each data in the detection set that is located after the second segmented data and the second segmented data are determined as a fourth detection interval.

6. The method according to claim 5, characterized in that After obtaining the first detection interval, the second detection interval, the third detection interval, and the fourth detection interval, the method further includes: Calling the interval statistical algorithm to process the detection set to obtain a statistical parameter group of the detection set; Acquire a first segmentation boundary based on the statistical parameter group of the detection set and each of the drift operation parameters; acquiring a second segmentation boundary based on the statistical parameter group of the first detection interval and each of the drift operation parameters; acquiring a third segmentation boundary based on the statistical parameter group of the third detection interval and each of the drift operation parameters; determining whether the first segmented data needs to be updated based on the first segmentation boundary, the second segmentation boundary, the statistical parameter group of the detection set, and the statistical parameter group of the first detection interval; and updating the target data to the first segmented data in the detection set if it is determined that the first segmented data needs to be updated; Based on the first segmentation boundary, the third segmentation boundary, the statistical parameter group of the detection set, and the statistical parameter group of the third detection interval, determine whether the second segmentation data needs to be updated; when it is determined that the second segmentation data needs to be updated, update the target data to the second segmentation data in the detection set.

7. A data processing model performance improvement device, characterized in that: include: a reading unit, configured to determine a test data stream of a data processing model, read each data in the test data stream one by one, and use the currently read data as target data; The data in the test data stream is electrocardiogram data; the data processing model is used to classify the electrocardiogram data; A first acquisition unit is configured to acquire various anomaly detection parameters pre-generated in the detection module; each of the anomaly detection parameters includes an upper edge parameter, a lower edge parameter, and a minimum anomaly ratio parameter; a detection unit, configured to perform concept drift detection on the data processing model based on the target data and each of the anomaly detection parameters; a triggering unit, configured to trigger the detection module to update various anomaly detection parameters and retrain the data processing model when determining that concept drift occurs in the data processing model; The return execution unit is configured to, after the data processing model training is completed, use the data read next time as the target data, obtain the updated anomaly detection parameters in the detection module, and return to execute the operation of performing concept drift detection on the data processing model based on the target data and the anomaly detection parameters until all data in the test data stream are read; Using the retrained data processing model to classify the electrocardiogram data, and obtaining the classification results of the electrocardiogram data; Determine an upper edge parameter and a lower edge parameter in each anomaly detection parameter, and determine the upper edge parameter, the lower edge parameter and a preset warning confidence as warning operation parameters; Processing the statistical parameter group of the first detection interval, the statistical parameter group of the second detection interval, and each early warning operation parameter to obtain a first early warning boundary value and a first early warning detection value; Processing the statistical parameter group of the third detection interval, the statistical parameter group of the fourth detection interval, and each early warning operation parameter to obtain a second early warning boundary value and a second early warning detection value; When the first warning detection value is greater than the first warning boundary value, and / or the second warning detection value is greater than the second warning boundary value, determining that the data processing model meets the warning condition; When the first warning detection value is not greater than the first warning boundary value, and the second warning detection value is not greater than the second warning boundary value, it is determined that the data processing model does not meet the warning conditions; when the data processing model meets the warning conditions, a warning message is sent; the first detection interval, the second detection interval, the third detection interval and the fourth detection interval are obtained by segmenting the data in the detection set; the detection set is used to save the data read from the test data stream; the statistical parameter group of the first detection interval, the second detection interval, the third detection interval and the fourth detection interval is obtained by using a preset interval statistical algorithm to process the first detection interval, the second detection interval, the third detection interval and the fourth detection interval respectively, and the statistical parameter group includes an exponentially weighted moving average and a step value.

8. A storage medium, characterized in that: The storage medium includes stored instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to execute the data processing model performance improvement method according to any one of claims 1 to 6.

9. An electronic device, characterized in that: The method comprises a memory and one or more instructions, wherein the one or more instructions are stored in the memory and configured to be executed by one or more processors to perform the data processing model performance improvement method according to any one of claims 1 to 6.

Images