Data correction method, device, electronic device and storage medium

Abstract

The embodiments of the present application provide a data correction method, device, electronic device and storage medium, which relate to the field of data processing technology. The method includes: in response to a correction trigger instruction, obtaining target data within a preset statistical period and reference data within a historical statistical period, wherein the target data includes order data that characterizes abnormal model prediction values, and the reference data refers to order data whose model prediction values ​​meet preset requirements; based on the target data and the reference data, generating negative sample data corresponding to the reference data, wherein the distribution of abnormal data in the negative sample data matches the distribution of abnormal data in the target data; generating correction data based on the prediction values ​​corresponding to the negative sample data and the reference data respectively; and correcting the prediction values ​​of the model based on the correction data. The embodiments of the present application can quickly locate abnormal features and improve the accuracy of data correction.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of data processing technology, and in particular to a data correction method, device, electronic device and storage medium. Background Art

[0002] In order to improve the user experience, during the instant delivery stage in online shopping scenarios, the delivery time is estimated through a time estimation model so that users can know the delivery period of their items.

[0003] When the time estimated by the time estimation model deviates greatly from the actual delivery time, the staff usually tries to correct the deviation of the estimated time output by the model by tentatively modifying the parameters of the time estimation model. However, this manual trial method is not only inefficient in correction, but also unable to determine the factors that cause the output anomaly, resulting in low accuracy of the correction results, which seriously affects the user experience. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the embodiments of the present application provide a data correction method, device, electronic device and storage medium.

[0005] According to a first aspect of an embodiment of the present application, a data correction method is provided, the method comprising: in response to a correction trigger instruction, obtaining target data within a preset statistical period and reference data within a historical statistical period, the target data comprising order data characterizing abnormalities in model prediction values, and the reference data referring to order data whose model prediction values ​​meet preset requirements; generating negative sample data corresponding to the reference data based on the target data and the reference data, the distribution of abnormal data in the negative sample data matching the distribution of abnormal data in the target data; generating correction data based on the prediction values ​​respectively corresponding to the negative sample data and the reference data; and correcting the prediction values ​​of the model based on the correction data.

[0006] According to a second aspect of an embodiment of the present application, a data correction device is provided, the device comprising: a data acquisition module, for acquiring target data within a preset statistical period and reference data within a historical statistical period in response to a correction trigger instruction, the target data comprising order data characterizing abnormalities in model prediction values, and the reference data referring to order data whose model prediction values ​​meet preset requirements; a negative sample generation module, for generating negative sample data corresponding to the reference data based on the target data and the reference data, the distribution of abnormal data in the negative sample data matching the distribution of abnormal data in the target data; a correction data generation module, for generating correction data based on the prediction values ​​respectively corresponding to the negative sample data and the reference data; and an output module, for correcting the prediction values ​​of the model based on the correction data.

[0007] According to a third aspect of an embodiment of the present application, there is provided an electronic device, comprising a non-volatile memory, a processor, a register, and a computer program stored on the register and executable on the processor, wherein the computer program is executed by the processor to enable the electronic device to read and write data to the non-volatile memory, and to execute the method described in the first aspect.

[0008] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored, and the program is executed by a processor to implement the method described in the first aspect.

[0009] According to a fifth aspect of an embodiment of the present application, a computer program product is provided, comprising instructions, which, when executed on a computer, cause the computer to execute the method according to the first aspect.

[0010] The technical solution provided by the embodiments of the present application may have the following beneficial effects:

[0011] In the embodiment of the present application, negative sample data corresponding to the reference data is generated based on the target data and reference data within a preset statistical period, the target data includes order data that characterizes the abnormality of the model prediction value, and the reference data refers to order data whose model prediction value meets the preset requirements, that is, based on the order data with abnormal prediction values, negative sample data is simulated in the normal reference data, wherein the distribution of abnormal data in the simulated negative sample data matches the distribution of abnormal data in the target data, which can improve the simulation accuracy of abnormal data and quantitatively determine the impact of abnormal features on the module. Then, according to the prediction values ​​corresponding to the negative sample data and the reference data, the correction data is generated; the prediction value of the model is corrected according to the correction data, which has the effect of quickly locating abnormal features and improving the accuracy of data correction.

[0012] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments of the present application. It should be understood that for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor.

[0014] Figure 1 A flowchart of the steps of the data correction method provided in the embodiment of the present application;

[0015] Figure 2 A flowchart of the steps for generating negative sample data provided in an embodiment of the present application;

[0016] Figure 3 A schematic diagram of the structure of a data correction device provided in an embodiment of the present application;

[0017] Figure 4 An exemplary structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0018] The technical solution of the embodiment of the present application is described below in conjunction with the drawings in the embodiment of the present application.

[0019] The terms used in the following embodiments of the present application are for the purpose of describing specific embodiments, and are not intended to be used as limitations on the technical solutions of the present application. As used in the specification and appended claims of the present application, the singular expressions "a", "an", "said", "above", "the" and "this" are intended to include plural expressions as well, unless there is a clear indication to the contrary in the context.

[0020] It should also be understood that although the terms first, second, etc. may be used to describe a certain type of object in the following embodiments, the object should not be limited to these terms. These terms are used to distinguish the specific implementation objects of this type of object.

[0021] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of the present application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0022] In online shopping scenarios, after users select products and place orders, the generated orders are assigned to delivery personnel, who then perform the delivery tasks. Delivery personnel can be riders, couriers, and self-mobile delivery equipment such as robots. In actual applications, the delivery system will estimate the delivery time through a time estimation model so that users can know the delivery time of their items.

[0023] However, in actual applications, there may be emergencies such as weather and accidents that affect the actual delivery time. When the actual delivery time is inconsistent with the estimated time, it will affect the user experience. When the emergency situation has not yet been restored, the estimated value of the time estimation model needs to be corrected to make it consistent with the current situation and realize emergency correction of the data. At present, when the estimated time of the time estimation model deviates greatly from the actual delivery time, the staff usually tries to correct the deviation of the estimated time output by the model by tentatively modifying the parameters of the time estimation model. However, this manual trial method is not only inefficient in correction efficiency, but also unable to determine the factors that cause the output abnormality, resulting in low accuracy of the correction result, which seriously affects the user experience.

[0024] For this, the present embodiment provides a data correction method, which generates negative sample data corresponding to the reference data according to the target data and reference data within a preset statistical period, wherein the target data includes order data that characterizes the abnormality of the model prediction value, and the reference data refers to order data whose model prediction value meets the preset requirements, that is, based on the order data with abnormal prediction value, the negative sample data is simulated in the normal reference data, wherein the distribution of abnormal data in the simulated negative sample data matches the distribution of abnormal data in the target data, which can improve the simulation accuracy of the abnormal data and quantitatively determine the impact of the abnormal features on the module. Then, according to the prediction values ​​corresponding to the negative sample data and the reference data, the correction data is generated; the prediction value of the model is corrected according to the correction data, which has the effect of quickly locating the abnormal features and improving the accuracy of data correction.

[0025] The following is a detailed description of a data correction method provided by an embodiment of the present application in conjunction with the accompanying drawings. In one embodiment, Figure 1 As shown, a flowchart of a data correction method is provided, and the method is described by taking the method applied to a computer device as an example, including the following steps S101 to S104:

[0026] S101 . In response to a deviation correction trigger instruction, obtain target data within a preset statistical period and reference data within a historical statistical period.

[0027] In this embodiment, the correction trigger instruction is, for example, a user clicking a trigger button on the interactive interface to trigger the start of the correction program. In one example, the target data and the reference data can be data actively uploaded by the user, or can be obtained from the historical log when the computer device retrieves the historical log after receiving the correction trigger instruction.

[0028] The user issues a correction trigger instruction, which indicates that the estimated time of the current time prediction model deviates greatly from the actual delivery time. Therefore, the target data includes order data that indicates abnormal model prediction values, and the reference data refers to order data whose model prediction values ​​meet preset requirements. In an example, meeting preset requirements may include on-time delivery of orders.

[0029] Order data includes order time, order delivery address, user information, product category, merchant information, delivery person information, etc. Some other order data information needs to be calculated and missing value filled before it can be obtained, such as the delivery person's stress coefficient, the delivery person's historical order volume in a week, etc.

[0030] The preset statistical period can be every one hour, two hours, three hours, etc., and the historical statistical period is the period corresponding to the preset statistical period. For example, if the statistical period of the order data with abnormal model prediction values ​​is from 10 to 11 am today, then the historical statistical period can be from 10 to 11 am yesterday, or from 10 to 11 am the day before yesterday.

[0031] S102: Generate negative sample data corresponding to the reference data according to the target data and the reference data, wherein the distribution of abnormal data in the negative sample data matches the distribution of abnormal data in the target data.

[0032] In this embodiment, the target data includes order data that characterizes abnormal model prediction values. Therefore, through feature analysis, the distribution of each feature data in the target data can be obtained, and then the distribution of abnormal data in the target data can be obtained. Similarly, the distribution of each feature data in the reference data can also be obtained.

[0033] The reference data of this embodiment refers to the order data whose model prediction value meets the preset requirements, that is, the positive sample data. Then, according to the target data and the reference data, the negative sample data corresponding to the reference data is generated, that is, the order data in the reference data that meets the preset requirements is modified into abnormal data to obtain the negative sample data. In order to make the obtained negative sample data more in line with the abnormal situation, the distribution of the abnormal data in the negative sample data is matched with the distribution of the abnormal data in the target data. The matching can be the same distribution ratio, or it can be close within the error range, or it can be a preset ratio. For example, if the abnormal data accounts for 70% in the target data, 70% of the order data in the reference data is also modified into abnormal data.

[0034] S103: Generate correction data according to the prediction values ​​corresponding to the negative sample data and the reference data.

[0035] Specifically, the negative sample data is obtained based on the reference data, that is, the negative sample data changes only in the distribution of abnormal data relative to the reference data. Then, the time prediction model is called to use the negative sample data and the reference data as the input of the model respectively, and the prediction values ​​corresponding to the negative sample data and the reference data respectively can be obtained. Through the difference between these two prediction values, the influence value of the negative sample data on the model can be obtained, that is, the influence of the abnormal data on the model prediction value. Then, according to the prediction values ​​corresponding to the negative sample data and the reference data respectively, the correction data can be generated. The correction data can be the difference between the prediction values ​​corresponding to the negative sample data and the reference data respectively.

[0036] S104: Correct the predicted value of the model according to the correction data.

[0037] Specifically, after the correction data is obtained, before the time prediction model performs time prediction for the next time period, the correction data can be used as an adjustment amount for the output value of the time prediction model, thereby correcting the prediction value of the model.

[0038] The data correction method provided in this embodiment generates negative sample data corresponding to the reference data according to the target data and reference data within a preset statistical period, wherein the target data includes order data that characterizes the abnormality of the model prediction value, and the reference data refers to order data whose model prediction value meets the preset requirements, that is, based on the order data with abnormal prediction value, the negative sample data is simulated in the normal reference data, wherein the distribution of abnormal data in the simulated negative sample data matches the distribution of abnormal data in the target data, which can improve the simulation accuracy of the abnormal data and quantitatively determine the impact of the abnormal features on the module. Then, according to the prediction values ​​corresponding to the negative sample data and the reference data, the correction data is generated; the prediction value of the model is corrected according to the correction data, which has the effect of quickly locating the abnormal features and improving the accuracy of data correction.

[0039] In one embodiment, both the target data and the reference data include multiple order data, which include order time, order delivery address, user information, meal delivery time, rider speed, product category, and other order information. In other words, any order data includes feature data of multiple feature dimensions. Figure 2 Step S102 can be specifically implemented by the following method:

[0040] S201. For each feature dimension, determine distribution parameters of feature data of the feature dimension in target data and reference data respectively.

[0041] S202. Determine the abnormal features that cause the model prediction value to be abnormal according to the distribution parameters corresponding to the feature parameters of multiple feature dimensions.

[0042] S203: Generate negative sample data according to abnormal features and reference data.

[0043] Specifically, assuming that both the target data and the reference data include 100 order data, then for each feature dimension in the target data and the reference data, there are 100 feature data. For example, in one order data, the meal delivery time is 10 minutes, and in another order data, the meal delivery time is 15 minutes. Then, the distribution of the feature dimension of meal delivery time in these 100 order data can be statistically obtained. Therefore, the distribution parameters of the feature data of each feature dimension in the target data and the reference data can be statistically obtained. The distribution parameters can be obtained through the probability distribution of the feature data of each feature dimension in the target data and the reference data.

[0044] The abnormal features that cause the model prediction value to be abnormal are determined based on the distribution parameters corresponding to the feature parameters of multiple feature dimensions. That is, the distribution parameters of the feature data of each feature dimension in the target data and the reference data are calculated one by one. According to the difference between the two distribution parameters of different feature data of the same feature dimension in the target data and the reference data, the abnormal features that cause the model prediction value to be abnormal can be determined. For example, if the feature parameter corresponding to a feature dimension in the target data is A and the feature parameter corresponding to the reference data is B, it means that the distribution of the feature dimension in the target data and the reference data is inconsistent. The abnormal features that cause the model prediction value to be abnormal can be determined by comparing the difference between A and B. For example, when the difference between A and B is large, it is considered that the feature dimension is an abnormal feature that causes the model prediction value to be abnormal.

[0045] In this embodiment, the characteristic parameters of each characteristic dimension may also be parameters such as expectation, variance, standard deviation, etc. that can describe information such as the shape, position, and dispersion of the probability distribution of the characteristic data in the whole.

[0046] After obtaining the abnormal features, the feature data corresponding to the abnormal features can be adjusted in the reference data based on the abnormal features and the reference data, that is, the abnormal data can be simulated in the order data that meets the preset requirements, thereby generating negative sample data.

[0047] The above embodiment can determine the abnormal features that cause abnormal model prediction values ​​according to the probability distribution parameters of each feature dimension, generate negative sample data according to the abnormal features and reference data, and simulate the distribution of abnormal features in normal data. It can quantitatively determine the impact of abnormal features on the model and improve the accuracy of correction data.

[0048] In one embodiment, the distribution parameters corresponding to each feature dimension include a first proportion and a second proportion, the first proportion refers to the proportion of feature data of the feature dimension in the target data, and the second proportion refers to the proportion of feature data of the feature dimension in the reference data; the abnormal features that cause the model prediction value to be abnormal are determined according to the distribution parameters corresponding to the feature parameters of multiple feature dimensions, including: for each feature dimension, calculating the difference between the first proportion and the second proportion corresponding to the feature dimension; determining the feature composed of the feature data of the feature dimension corresponding to the difference greater than a preset threshold as an abnormal feature.

[0049] In one example, the distribution parameter of the feature data of a feature dimension in the target data can be represented by F1(x), and the distribution parameter of the feature data of a feature dimension in the reference data can be represented by F2(x). F2(1) and F2(x) are the cumulative distribution functions of two probability distributions. Furthermore, F1(x) can be used to represent the first proportion, and F2(x) can be used to represent the second proportion. According to the difference between the first proportion and the second proportion, the Kolmogorov-Smirnov statistic (KS) can be used to represent it. The KS indicator can be used to measure the degree of difference between two samples, that is, KS = max|F1(x)-F2(x)|.

[0050] Then, a preset threshold can be set, and the feature data of the feature dimension corresponding to the difference greater than the preset threshold is determined as an abnormal feature. For example, in this embodiment, there are 80 feature dimensions, and KS1, KS2, ... KS3 can be obtained. 79 , KS 80 There are 80 KS values ​​in total, of which 4 KS values ​​are greater than the preset threshold. The feature dimensions corresponding to these 4 KS values ​​are: merchant meal preparation time, rider delivery speed, weather reasons and traffic congestion. The four features of merchant meal preparation time, rider delivery speed, weather reasons and traffic congestion are determined as abnormal features.

[0051] Optionally, the preset threshold can be 0.2, or multiple preset thresholds can be set, such as setting a first preset threshold and a second preset threshold, the first preset threshold is less than the second preset threshold, when the KS value is greater than the first threshold, the feature corresponding to the KS value can be determined as an abnormal feature, when the KS value is greater than the second threshold, the feature corresponding to the KS value can be determined as a key abnormal feature, that is, a feature that has a greater impact on the output value of the model. The specific size of the preset threshold can be customized according to the actual calculation accuracy, and is not specifically limited here.

[0052] This embodiment uses a feature probability density distribution method to determine the distribution difference between feature data in target data with an abnormality and feature data in reference data of the same historical period, so as to quickly locate the abnormal feature.

[0053] In one embodiment, negative sample data is generated based on abnormal features and reference data, including: determining a first ratio of feature data in the feature data of the abnormal features that meets a first condition to all feature data under the same feature dimension, the first condition including that the feature data is feature data obtained by replacing at least one of missing values ​​and fill-in values; modifying feature data in the reference data with a dimension matching the abnormal features into abnormal data to obtain negative sample data, wherein the proportion of feature data in the abnormal data in the negative sample data that meets the first condition matches the first ratio.

[0054] In one example, before using sample data to calculate probability distribution, the data needs to be processed and analyzed. When missing, incomplete or abnormal data is encountered, default values ​​are usually used to replace it. For example, when the delivery time in an order data is missing, statistics such as mean, median, mode, or simple fixed values ​​(such as 0 or a specific value) can be used to fill these missing data points. In other words, the order data in the target data and reference data are obtained by processing the default values ​​of the data. Among them, the distribution of default values ​​can characterize the proportion of abnormal data in an order data. For example, when there are 100 values ​​in an order data, 70% of which are default values, then 70% of the data in the order data are considered to be abnormal values. Then, 70% of the data in the feature data of the same feature dimension in the reference data can also be modified as abnormal data, and negative sample data obtained by simulating the reference data under the same abnormal feature dimension can be obtained.

[0055] For example, if the feature data of abnormal feature A has 100 values, and the default value accounts for 60%, that is, the first ratio is 60%, then 60% of the feature data of the dimension where abnormal feature A is located in the reference data is also modified to the default value. For each dimension of the abnormal feature, the above operation is performed to generate negative sample data based on the abnormal feature and the reference data. Among them, the proportion of feature data that meets the first condition in the abnormal data in the negative sample data matches the first ratio, which can improve the simulation accuracy of the negative sample data.

[0056] This embodiment modifies the feature data in the reference data according to the proportion of abnormal data in the feature data of the abnormal feature to obtain negative sample data, so that the proportion of abnormal data in the negative sample data matches the proportion of abnormal data in the target data, which can improve the simulation accuracy of the negative sample data.

[0057] In one example, data in a dimension matching an abnormal feature in reference data is modified into abnormal data, including: for each feature dimension of the abnormal feature, a simulation threshold is calculated based on a first ratio corresponding to the feature dimension, and a second ratio of feature data satisfying a first condition under the feature dimension in the reference data to all feature data; order data in the reference data having the same dimension as the abnormal feature is converted into a random number, wherein the random number includes a decimal between 0 and 1; and when the random number is less than or equal to the simulation threshold, the order data corresponding to the random number is modified into abnormal data.

[0058] For example, the target data and the reference data both include 100 order data, the abnormal feature is A, the first ratio of A to the default value in the target data is a, and the ratio of A to the default value in the reference data is b, that is, the second ratio of the feature data set to the default value under the feature dimension A in the reference data to all 100 feature data under the feature dimension A is b. Then the simulation threshold th=a(1-b) can be obtained based on a and b, and th is used as the threshold. The 100 feature data of the feature dimension A in the reference data are converted into random numbers, rand, rand is uniformly distributed between [0,1]. When rand<=th, the feature data corresponding to the feature dimension A in the reference data is modified to the default value, otherwise the original value of the feature data corresponding to the feature dimension A is maintained, and negative sample data can be simulated in the reference data.

[0059] This embodiment uses a known abnormal distribution parameter, that is, a first ratio corresponding to a feature dimension, and a random algorithm to simulate the distribution of abnormal features in reference data, so as to simulate negative sample data with a specified abnormal distribution, thereby improving the simulation accuracy of the negative sample data.

[0060] In one example, correction data is generated based on the prediction values ​​corresponding to the negative sample data and the reference data, including: calling the model to predict the negative sample data to obtain a first prediction value, and calling the model to predict the reference data to obtain a second prediction value; and using the difference between the first prediction value and the second prediction value as the correction data.

[0061] It can be seen from the above embodiments that the negative sample data is obtained by simulating the distribution of abnormal data corresponding to the abnormal features in the target data on the basis of the reference data. That is to say, the difference between the negative sample data and the reference data lies only in the different distribution of the abnormal data corresponding to the abnormal features. Therefore, the model can be called to predict the negative sample data and the reference data respectively to obtain the first prediction value and the second prediction value, and then the difference between the first prediction value and the second prediction value can be compared to obtain the correction data.

[0062] For example, if the difference between the first prediction value and the second prediction value is 200s, it means that the abnormal feature occurring in the target data will cause the model to have a time prediction value deviation of 200s, and the 200s is used as correction data.

[0063] In an example, the first prediction value and the second prediction value may be data values ​​of the model, or may be observation indicators of the model, such as mean square error (MAE).

[0064] This embodiment uses the model to predict the negative sample data and the reference data respectively, so that the data variables can be controlled to the distribution of abnormal data corresponding to a single abnormal feature, thereby improving the calculation accuracy of the correction data.

[0065] In one example, it can be seen from the above embodiments that when a user issues a correction trigger instruction, the estimated time of the current time prediction model deviates greatly from the actual delivery time. In this case, the user can know whether the estimated time of the current time prediction model is greater than the actual delivery time or less than the actual delivery time. Based on this, the user's correction intervention instruction can be to add correction data to the model prediction value of the time prediction model, or to subtract correction data from the model prediction value of the time prediction model.

[0066] Specifically, when the estimated time of the time prediction model is greater than the actual delivery time, the user's correction intervention instruction is to subtract the initial prediction value from the correction data; when the estimated time of the time prediction model is less than the actual delivery time, the user's correction intervention instruction is to add the initial prediction value to the correction data.

[0067] In this embodiment, the prediction value of the model is corrected according to the correction data, including: obtaining the initial prediction value of the model based on current data, the current data including current order data; in response to the correction intervention instruction, adding or subtracting the correction data from the initial prediction value to obtain the prediction value after the initial prediction value is corrected; and outputting the corrected prediction value.

[0068] Specifically, after obtaining the correction data as mentioned above, when the delivery time is predicted again based on the model and the current data to obtain the model prediction value, the initial prediction value can be added or subtracted from the correction data by receiving the user's correction intervention instruction to obtain the prediction value after correction of the initial prediction value.

[0069] In one example, the current data includes current order data, and the current order data has the same feature dimension as the feature data in the reference data.

[0070] For example, the initial prediction value is 600s and the correction data is 200s. If the correction intervention instruction is to add the initial prediction value to the correction data, the output prediction value after correction is 800s. If the correction intervention instruction is to subtract the initial prediction value from the correction data, the output prediction value after correction is 400s.

[0071] This embodiment can achieve the effect of quickly correcting the model output value without modifying the parameters of the model itself. That is to say, by adding or subtracting the correction data obtained by the above method of this embodiment from the model's own output value, the model output value can be quickly corrected.

[0072] The above embodiment provides a data correction method, which generates negative sample data corresponding to the reference data according to the target data and reference data within a preset statistical period, wherein the target data includes order data that characterizes the abnormality of the model prediction value, and the reference data refers to order data whose model prediction value meets the preset requirements, that is, based on the order data with abnormal prediction value, the negative sample data is simulated in the normal reference data, wherein the distribution of abnormal data in the simulated negative sample data matches the distribution of abnormal data in the target data, which can improve the simulation accuracy of abnormal data and quantitatively determine the impact of abnormal features on the module. Then, according to the prediction values ​​corresponding to the negative sample data and the reference data, the correction data is generated; the prediction value of the model is corrected according to the correction data, which has the effect of quickly locating abnormal features, and can achieve the effect of quickly correcting the model output value without modifying the parameters of the model itself, thereby improving the accuracy of data correction.

[0073] If the implementation steps of the above embodiment can realize the corresponding functions through a software module, corresponding to the above data correction method, the embodiment of the present application can also provide a data correction device.

[0074] like Figure 3 As shown, a schematic diagram of the structure of a data correction device is provided. The data correction device 300 may include a data acquisition module 301, a negative sample generation module 302, a correction data generation module 303, and an output module 304. The data correction device 300 may be used to perform the above Figure 1 to Figure 2 operation of part or all of the electronic equipment in the

[0075] For example: the data acquisition module 301 is used to respond to the correction trigger instruction to obtain the target data within a preset statistical period and the reference data within a historical statistical period, wherein the target data includes order data representing abnormal model prediction values, and the reference data refers to order data whose model prediction values ​​meet preset requirements.

[0076] The negative sample generation module 302 is used to generate negative sample data corresponding to the reference data according to the target data and the reference data, and the distribution of abnormal data in the negative sample data matches the distribution of abnormal data in the target data.

[0077] The correction data generation module 303 is used to generate correction data according to the prediction values ​​corresponding to the negative sample data and the reference data respectively.

[0078] The output module 304 is used to correct the prediction value of the model according to the correction data.

[0079] It can be seen that the data correction device 300 provided in the embodiment of the present application has the effect of quickly locating abnormal features and improving the accuracy of data correction.

[0080] Optionally, any order data includes feature data of multiple feature dimensions, and the negative sample generation module 302 is used to determine, for each feature dimension, the distribution parameters of the feature data of the feature dimension in the target data and the reference data respectively; determine the abnormal features that cause the model prediction value to be abnormal according to the distribution parameters corresponding to the feature parameters of the multiple feature dimensions respectively; and generate the negative sample data according to the abnormal features and the reference data.

[0081] Optionally, the distribution parameters corresponding to each feature dimension include a first proportion and a second proportion, wherein the first proportion refers to the proportion of the feature data of the feature dimension in the target data, and the second proportion refers to the proportion of the feature data of the feature dimension in the reference data. The negative sample generation module 302 is used to calculate the difference between the first proportion and the second proportion corresponding to each feature dimension; and determine the feature composed of the feature data of the feature dimension corresponding to the difference greater than a preset threshold as the abnormal feature.

[0082] Optionally, the negative sample generation module 302 is used to determine a first ratio of feature data in the feature data of the abnormal feature that meets a first condition to all feature data under the same feature dimension, wherein the first condition includes that the feature data is feature data obtained by replacing at least one of missing values ​​and filled values; the feature data of the dimension matching the abnormal feature in the reference data is modified into abnormal data to obtain the negative sample data, and the proportion of feature data in the abnormal data in the negative sample data that meets the first condition matches the first ratio.

[0083] Optionally, the negative sample generation module 302 is used to calculate a simulation threshold for each feature dimension of the abnormal feature according to a first ratio corresponding to the feature dimension and a second ratio of feature data that satisfies the first condition under the feature dimension in the reference data to all feature data; convert order data in the reference data with the same dimension as the abnormal feature into a random number, wherein the random number includes a decimal between 0 and 1; and when the random number is less than or equal to the simulation threshold, modify the order data corresponding to the random number to abnormal data.

[0084] Optionally, the correction data generation module 303 is used to call the model to predict the negative sample data to obtain a first prediction value, and call the model to predict the reference data to obtain a second prediction value; and use the difference between the first prediction value and the second prediction value as the correction data.

[0085] Optionally, the output module 304 is used to obtain an initial prediction value of the model based on current data, where the current data includes current order data; in response to a correction intervention instruction, add or subtract the initial prediction value from the correction data to obtain a prediction value after correction of the initial prediction value; and output the corrected prediction value.

[0086] Understandably, Figure 3 The division of each module in the above is only a division of logical functions. In actual implementation, the functions of the above modules can be integrated into the hardware entity of the electronic device. Please refer to Figure 4 , Figure 4 An electronic device is provided, the electronic device 80 comprising: a processor 800, a memory 801, a bus 802 and a communication interface 803, the processor 800, the communication interface 803 and the memory 801 being connected via the bus 802; the memory 801 storing a computer program that can be run on the processor 800, and the processor 800 executes the method provided in any of the aforementioned embodiments of the present application when running the computer program.

[0087] The memory 801 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 803 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0088] The bus 802 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The memory 801 is used to store programs, and the processor 800 executes the programs after receiving the execution instruction. The data deflection method disclosed in any implementation of the embodiment of the present application may be applied to the processor 800, or implemented by the processor 800.

[0089] The processor 800 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 800. The above processor 800 can be a general-purpose processor, including a processor (Central Processing Unit, referred to as CPU), a network processor (Network Processor, referred to as NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a readily available programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor to be executed, or the hardware and software modules in the decoding processor can be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 801, and the processor 800 reads the information in the memory 801 and completes the steps of the above method in combination with its hardware.

[0090] A computer-readable storage medium is also provided in an embodiment of the present application. The computer-readable storage medium stores instructions for reading and writing data. When the computer-readable storage medium is run on a computer, the computer executes part or all of the steps in the method described in the above-mentioned embodiment.

[0091] The embodiment of the present application also provides a computer program product including instructions for reading and writing data, which, when executed on a computer, enables the computer to execute part or all of the steps in the method described in the above-mentioned embodiment.

[0092] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0093] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0094] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0095] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0096] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a smart phone or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0097] Although the optional embodiments of the present application have been described, those skilled in the art, once informed of the basic creative concept, may make additional changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiment and all changes and modifications falling within the scope of the present application.

[0098] The specific implementation methods described above further illustrate the purpose, technical solutions and beneficial effects of the present application in detail. It should be understood that the above description is only the specific implementation method of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the present application should be included in the scope of protection of the present invention.

Claims

1. A data correction method, characterized in that: The method comprises: In response to the deviation correction trigger instruction, target data within a preset statistical period and reference data within a historical statistical period are obtained, wherein the target data includes order data representing abnormal model prediction values, and the reference data refers to order data whose model prediction values ​​meet preset requirements; Generate negative sample data corresponding to the reference data according to the target data and the reference data, wherein the distribution of abnormal data in the negative sample data matches the distribution of abnormal data in the target data; Generate correction data according to the prediction values ​​corresponding to the negative sample data and the reference data respectively; The predicted value of the model is corrected according to the correction data.

2. The data correction method according to claim 1, characterized in that: Any order data includes feature data of multiple feature dimensions, and generating negative sample data corresponding to the reference data according to the target data and the reference data includes: For each feature dimension, determining distribution parameters of the feature data of the feature dimension in the target data and the reference data respectively; Determining the abnormal features that cause the model prediction value to be abnormal according to the distribution parameters respectively corresponding to the feature parameters of the multiple feature dimensions; The negative sample data is generated according to the abnormal feature and the reference data.

3. The data correction method according to claim 2, characterized in that: The distribution parameters corresponding to each feature dimension include a first proportion and a second proportion, wherein the first proportion refers to the proportion of the feature data of the feature dimension in the target data, and the second proportion refers to the proportion of the feature data of the feature dimension in the reference data; the determining of the abnormal features that cause the abnormality of the model prediction value according to the distribution parameters corresponding to the feature parameters of the multiple feature dimensions respectively includes: For each feature dimension, calculating the difference between the first proportion and the second proportion corresponding to the feature dimension; The feature composed of the feature data of the feature dimension corresponding to the difference greater than the preset threshold is determined as the abnormal feature.

4. The data correction method according to claim 2, characterized in that: Generating the negative sample data according to the abnormal feature and the reference data includes: Determine a first ratio of feature data satisfying a first condition in the feature data of the abnormal feature to all feature data under the same feature dimension, wherein the first condition includes that the feature data is feature data obtained by replacing at least one of a missing value and a filled value; The feature data of the dimension matching the abnormal feature in the reference data is modified into abnormal data to obtain the negative sample data, and the proportion of the feature data satisfying the first condition in the abnormal data in the negative sample data matches the first ratio.

5. The data correction method according to claim 4, characterized in that: Modifying the data of the dimension matching the abnormal feature in the reference data into abnormal data includes: For each feature dimension of the abnormal feature, a simulation threshold is calculated according to a first ratio corresponding to the feature dimension and a second ratio of feature data satisfying the first condition under the feature dimension in the reference data to all feature data; Converting the order data of the same dimension as the abnormal feature in the reference data into a random number, wherein the random number includes a decimal between 0 and 1; When the random number is less than or equal to the simulation threshold, the order data corresponding to the random number is modified to abnormal data.

6. The data correction method according to claim 1, characterized in that: Generating correction data according to the prediction values ​​respectively corresponding to the negative sample data and the reference data, including: Calling the model to predict the negative sample data to obtain a first prediction value, and calling the model to predict the reference data to obtain a second prediction value; The difference between the first predicted value and the second predicted value is used as the correction data.

7. The data correction method according to claim 6, characterized in that: Correcting the predicted value of the model according to the correction data includes: Obtaining an initial prediction value of the model based on current data, wherein the current data includes current order data; In response to the correction intervention instruction, the initial prediction value is added or subtracted from the correction data to obtain a prediction value after the correction of the initial prediction value; The predicted value after the correction is output.

8. A data correction device, characterized in that: The device comprises: A data acquisition module, configured to acquire target data within a preset statistical period and reference data within a historical statistical period in response to a correction trigger instruction, wherein the target data includes order data representing abnormal model prediction values, and the reference data refers to order data whose model prediction values ​​meet preset requirements; A negative sample generation module, configured to generate negative sample data corresponding to the reference data according to the target data and the reference data, wherein the distribution of abnormal data in the negative sample data matches the distribution of abnormal data in the target data; A correction data generation module, used to generate correction data according to the prediction values ​​corresponding to the negative sample data and the reference data respectively; An output module is used to correct the predicted value of the model according to the correction data.

9. The data correction device according to claim 8, characterized in that: Any of the order data includes feature data of multiple feature dimensions, and the negative sample generation module is used to determine, for each feature dimension, distribution parameters of the feature data of the feature dimension in the target data and the reference data respectively; determine abnormal features that cause the model prediction value to be abnormal according to the distribution parameters corresponding to the feature parameters of the multiple feature dimensions respectively; and generate the negative sample data according to the abnormal features and the reference data.

10. The data correction device according to claim 8, characterized in that: The distribution parameters corresponding to each feature dimension include a first proportion and a second proportion, wherein the first proportion refers to the proportion of the feature data of the feature dimension in the target data, and the second proportion refers to the proportion of the feature data of the feature dimension in the reference data. The negative sample generation module is used to calculate the difference between the first proportion and the second proportion corresponding to each feature dimension; and determine the feature composed of the feature data of the feature dimension corresponding to the difference greater than a preset threshold as the abnormal feature.

Images