Data verification method, device, processor, and electronic device
Patent Information
- Application Number
- CN202310364368.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-04-06
- Publication Date
- 2026-09-18
- Estimated Expiration
- 2043-04-06
AI Technical Summary
[0005]本申请的主要目的在于提供一种数据检验方法、装置、处理器以及电子设备,以解决相关技术中难以快速识别异常报送量的问题
[0016]This application employs the following steps: acquiring target consumption data generated by multiple customers within a target time period, and determining the type of the date that generated the target consumption data and the consumption data threshold associated with that type; acquiring the actual reporting volume generated within the target time period, wherein the actual reporting volume refers to the number of customers whose consumption data exceeds the consumption data threshold; inputting the target consumption data into a type-matching prediction model, outputting N predicted reporting volumes, wherein the prediction model is obtained by training a preset prediction model with M sets of sample data, each set of sample data including historical consumption data and historical reporting volumes; verifying the actual reporting volume using the N predicted reporting volumes, obtaining a first verification result, and determining the status of the actual reporting volume based on the first verification result, wherein the status of the actual reporting volume includes a normal status or an abnormal status, solving the problem of difficulty in quickly identifying abnormal reporting volumes in related technologies. By using the prediction model to verify the actual reporting volume and obtaining the verification result, the consumption data is identified using the verification result, thereby achieving the effect of effectively identifying the status of the reporting volume.
Smart Images

Figure CN116304951B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of financial technology, and more specifically, to a data verification method, apparatus, processor, and electronic device. Background Technology
[0002] To strengthen the fight against financial crimes, enhance the monitoring and management of overseas transactions, and maintain order in overseas transactions, financial institutions are required to report abnormal consumption information from the previous day on a daily basis. At the same time, they are required to monitor and control the consumption generated by users on the blacklists issued to financial institutions in real time.
[0003] To prevent blacklisted users from making unauthorized purchases overseas, existing technologies involve processing the data to be identified along a series of steps before uploading. However, this process is lengthy and involves multiple steps. Furthermore, problems at any stage of the data processing chain can lead to data not being identified in a timely manner or data omissions, allowing blacklisted users to make further unauthorized purchases.
[0004] There is currently no effective solution to the problem of difficulty in quickly identifying abnormal reporting volumes in related technologies. Summary of the Invention
[0005] The main objective of this application is to provide a data verification method, apparatus, processor, and electronic device to solve the problem of difficulty in quickly identifying abnormal reporting volumes in related technologies.
[0006] To achieve the above objectives, according to one aspect of this application, a data verification method is provided. The method includes: acquiring target consumption data generated by multiple customers within a target time period, and determining the type of the date that generated the target consumption data and the consumption data threshold associated with that type; acquiring the actual reporting volume generated within the target time period, wherein the actual reporting volume refers to the number of customers whose consumption data exceeds the consumption data threshold; inputting the target consumption data into a type-matching prediction model, and outputting N predicted reporting volumes, wherein the prediction model is obtained by training a preset prediction model using M sets of sample data, each set of sample data including historical consumption data and historical reporting volumes; verifying the actual reporting volume using the N predicted reporting volumes to obtain a first verification result, and determining the status of the actual reporting volume based on the first verification result, wherein the status of the actual reporting volume includes a normal status or an abnormal status.
[0007] Optionally, the date type for generating the target consumption data includes target days and non-target days. The threshold for consumption data associated with target days is greater than the threshold for consumption data associated with non-target days. Inputting the target consumption data into the type-matching prediction model includes: when the type is target day, inputting the target consumption data into a first prediction model, wherein the first prediction model is obtained by training a preset prediction model with M1 sets of first sample data, the first sample data including historical target day consumption data and historical target day reporting volume; when the type is non-target day, inputting the target consumption data into a second prediction model, wherein the second prediction model is obtained by training a preset prediction model with M2 sets of second sample data, the second sample data including historical non-target day consumption data and historical non-target day reporting volume, the sum of M1 and M2 being M.
[0008] Optionally, the prediction model is trained in the following way: obtain M historical consumption data and historical reporting volume to obtain M sets of sample data; train a preset prediction model using the M sets of sample data to obtain an initial prediction model; determine whether the evaluation index value of the initial prediction model meets the preset requirements, and if the evaluation index value does not meet the preset requirements, adjust the model parameters of the initial prediction model until the evaluation index value of the adjusted prediction model meets the preset requirements to obtain the prediction model. The evaluation index value is calculated from the historical predicted reporting volume input to the prediction model and the historical predicted reporting volume output by the prediction model. The evaluation index includes at least one of the following: mean squared error and coefficient of determination.
[0009] Optionally, the actual reporting volume is verified using N predicted reporting volumes to obtain a first verification result. The determination of the status of the actual reporting volume based on the first verification result includes: calculating the mean and variance of the N predicted reporting volumes, and using the mean and variance to calculate the test value; comparing the test value with the confidence level, and if the test value is greater than or equal to the confidence level, determining the status of the actual reporting volume as an undetermined state, and if the test value is less than the confidence level, determining the status of the actual reporting volume as a normal state.
[0010] Optionally, after determining the status of the actual reported quantity as pending when the test value is greater than or equal to the confidence level, the method further includes: issuing an abnormality prompt message and obtaining the communication information of the inspector; uploading the actual reported quantity to the inspector based on the communication information; receiving the second verification result from the inspector; determining the status of the actual reported quantity as abnormal when the first verification result and the second verification result are the same; and determining the status of the actual reported quantity as normal when the first verification result and the second verification result are different.
[0011] Optionally, after obtaining the actual reported number of customers whose consumption data generated within the target time period exceeds the consumption data threshold, the method further includes: uploading the actual reported number to the inspector if the actual reported number exceeds the preset reported number threshold; and receiving the third verification result from the inspector, wherein the third verification result represents the status of the actual reported number.
[0012] Optionally, obtaining target consumption data generated by multiple customers within a target time period includes: obtaining customers from the blacklist stored by the target organization to obtain multiple customers; and obtaining data generated by multiple customers within the target time period from the database of the target organization to obtain target consumption data.
[0013] To achieve the above objectives, according to another aspect of this application, a data verification apparatus is provided. The apparatus includes: a first acquisition unit, configured to acquire target consumption data generated by multiple customers within a target time period, and determine the type of the date that generated the target consumption data and a consumption data threshold associated with that type; a second acquisition unit, configured to acquire the actual reporting volume generated within the target time period, wherein the actual reporting volume refers to the number of customers whose consumption data exceeds the consumption data threshold; an input unit, configured to input the target consumption data into a type-matching prediction model, and output N predicted reporting volumes, wherein the prediction model is obtained by training a preset prediction model using M sets of sample data, each set of sample data including historical consumption data and historical reporting volumes; and a verification unit, configured to verify the actual reporting volume using the N predicted reporting volumes, obtain a first verification result, and determine the state of the actual reporting volume based on the first verification result, wherein the state of the actual reporting volume includes a normal state or an abnormal state.
[0014] According to another aspect of the present invention, a processor is also provided, which is used to run a program, wherein the program controls a device containing a non-volatile storage medium to perform a data verification method during runtime.
[0015] According to another aspect of the present invention, an electronic device is also provided, comprising one or more processors and a memory; the memory stores computer-readable instructions, and the processor is configured to execute the computer-readable instructions, wherein the computer-readable instructions execute a data verification method when they are run.
[0016] This application employs the following steps: acquiring target consumption data generated by multiple customers within a target time period, and determining the type of the date that generated the target consumption data and the consumption data threshold associated with that type; acquiring the actual reporting volume generated within the target time period, wherein the actual reporting volume refers to the number of customers whose consumption data exceeds the consumption data threshold; inputting the target consumption data into a type-matching prediction model, outputting N predicted reporting volumes, wherein the prediction model is obtained by training a preset prediction model with M sets of sample data, each set of sample data including historical consumption data and historical reporting volumes; verifying the actual reporting volume using the N predicted reporting volumes, obtaining a first verification result, and determining the status of the actual reporting volume based on the first verification result, wherein the status of the actual reporting volume includes a normal status or an abnormal status, solving the problem of difficulty in quickly identifying abnormal reporting volumes in related technologies. By using the prediction model to verify the actual reporting volume and obtaining the verification result, the consumption data is identified using the verification result, thereby achieving the effect of effectively identifying the status of the reporting volume. Attached Figure Description
[0017] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:
[0018] Figure 1 This is a flowchart of a data verification method provided according to an embodiment of this application;
[0019] Figure 2 This is a schematic diagram of a data verification system provided according to an embodiment of this application;
[0020] Figure 3 This is a schematic diagram of a data verification device provided according to an embodiment of this application;
[0021] Figure 4 This is a schematic diagram of an electronic device provided according to an embodiment of this application. Detailed Implementation
[0022] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.
[0023] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.
[0024] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0025] It should be noted that all information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) involved in this disclosure are information and data authorized by the user or fully authorized by all parties.
[0026] The present invention will now be described in conjunction with preferred implementation steps. Figure 1 This is a flowchart of a data verification method provided according to an embodiment of this application, such as... Figure 1 As shown, the method includes the following steps:
[0027] Step S101: Obtain target consumption data generated by multiple customers within the target time period, and determine the type of the date that generated the target consumption data and the consumption data threshold associated with that type.
[0028] Specifically, the target time period can include a certain day or a certain period of time. The customer refers to a consumer who is located overseas and has made purchases by means of credit card or other means. The target consumption data refers to the consumption data of the customer who made overseas purchases or cash withdrawals during a certain period of time, which can include the type of bank card that generated the purchase or cash withdrawal, the date of the purchase or cash withdrawal, and the amount of the purchase or cash withdrawal.
[0029] Since different consumption times lead to different consumption amounts, it's necessary to categorize dates into target days and non-target days to better determine if consumption data is abnormal. Target days refer to overseas promotional days, while non-target days are the remaining days excluding overseas promotional days. Therefore, different consumption data thresholds can be set for different dates. After determining the type of date for the target consumption data, the associated consumption data threshold can be determined, thus yielding the reported volume data.
[0030] Step S102: Obtain the actual reporting volume generated within the target time period, where the actual reporting volume refers to the number of customers whose consumption data exceeds the consumption data threshold.
[0031] Specifically, when the target time period is a non-target day, the system counts customers whose spending data within that time period exceeds the spending data threshold for non-target days, obtains the number of these customers, and determines this number as the actual reported quantity. For example, if it is detected that the spending amount generated by A customers within the target time period exceeds the spending data threshold, the actual reported quantity is set to A.
[0032] Step S103: Input the target consumption data into the type-matching prediction model and output N predicted reporting volumes. The prediction model is obtained by training a preset prediction model with M sets of sample data. Each set of sample data includes historical consumption data and historical reporting volumes.
[0033] Specifically, after determining the type of target consumption data, the type of prediction model is determined. The type of prediction model can include prediction models for target days and prediction models for non-target days. The prediction model can be an Extreme Learning Machine (ELM) prediction model, which is a machine learning system or method based on a feedforward neural network. The weights of the hidden layer nodes are determined randomly or manually.
[0034] Specifically, the acquired target consumption data is input into the trained prediction model, and N predictions are made to obtain N predicted reporting volumes. The prediction model is trained from multiple sets of historical consumption data and historical reporting volumes.
[0035] Step S104: Verify the actual reporting volume using N predicted reporting volumes to obtain a first verification result, and determine the status of the actual reporting volume based on the first verification result. The status of the actual reporting volume includes a normal status or an abnormal status.
[0036] Specifically, the hypothesis testing is performed by comparing the multiple predicted reporting volumes output by the prediction model with the actual reporting volumes uploaded within the target time period to determine whether there is a significant difference between the actual reporting volume and the predicted reporting volume. If the first verification result indicates that there is a significant difference between the time reporting volume and the predicted reporting volume, the status of the actual reporting volume is first determined to be a pending confirmation status. After the data is manually verified, the status of the actual reporting volume is further determined, that is, the normal status or the abnormal status.
[0037] The data verification method provided in this application involves acquiring target consumption data generated by multiple customers within a target time period, determining the type of the date that generated the target consumption data, and the consumption data threshold associated with that type; acquiring the actual reporting volume generated within the target time period, where the actual reporting volume refers to the number of customers whose consumption data exceeds the consumption data threshold; inputting the target consumption data into a type-matching prediction model, outputting N predicted reporting volumes, where the prediction model is obtained by training a preset prediction model with M sets of sample data, each set of sample data including historical consumption data and historical reporting volumes; verifying the actual reporting volume using the N predicted reporting volumes to obtain a first verification result, and determining the status of the actual reporting volume based on the first verification result, where the status of the actual reporting volume includes a normal status or an abnormal status, thus solving the problem of difficulty in quickly identifying abnormal reporting volumes in related technologies. By using the prediction model to verify the actual reporting volume and obtaining the verification result, the consumption data is identified using the verification result, thereby achieving the effect of effectively identifying the status of the reporting volume.
[0038] The prediction model includes multiple types. Optionally, in the data verification method provided in this application embodiment, the type of the date that generates the target consumption data includes target days and non-target days. The threshold of consumption data associated with the target day is greater than the threshold of consumption data associated with the non-target day. Inputting the target consumption data into the type-matching prediction model includes: when the type is target day, inputting the target consumption data into a first prediction model, wherein the first prediction model is obtained by training a preset prediction model with M1 sets of first sample data, and the first sample data includes historical target day consumption data and historical target day reporting volume; when the type is non-target day, inputting the target consumption data into a second prediction model, wherein the second prediction model is obtained by training a preset prediction model with M2 sets of second sample data, and the second sample data includes historical non-target day consumption data and historical non-target day reporting volume, and the sum of M1 and M2 is M.
[0039] Specifically, the target day can be the major promotional day "Black Friday", and non-target days can include the remaining days other than "Black Friday". Since customers will make a lot of purchases on the target day, the consumption data thresholds for the target day and non-target days are different, and the consumption data threshold for the target day is greater than that for the non-target day. Therefore, it is necessary to classify the prediction model into a prediction model for the target day, which is the first prediction model, and a prediction model for the non-target day, which is the second prediction model.
[0040] After identifying whether the target time period belongs to the target day or not, the acquired target consumption data is input into the corresponding prediction model. For example, when the target time period belongs to the target day, the target consumption data is input into the first prediction model, which then makes multiple predictions for the target consumption data; when the target time period belongs to a non-target day, the target consumption data is input into the second prediction model, which then makes multiple predictions for the target consumption data.
[0041] It should be noted that the prediction model needs to be trained using multiple sets of sample data. Each set of sample data includes historical consumption data and historical reporting volume for the corresponding date. The first prediction model is trained using historical consumption data and historical reporting volume for the target date, while the second prediction model is trained using historical consumption data and historical reporting volume for non-target dates. By using sample data from target days and non-target days to train the prediction models separately, the prediction accuracy of the prediction models can be improved.
[0042] During the training process, evaluation metrics are used to determine whether the prediction model has high prediction accuracy. Optionally, in the data verification method provided in this application embodiment, the prediction model is trained in the following way: M historical consumption data and historical reporting volume are obtained to obtain M sets of sample data; a preset prediction model is trained using the M sets of sample data to obtain an initial prediction model; it is determined whether the evaluation metric value of the initial prediction model meets the preset requirements, and if the evaluation metric value does not meet the preset requirements, the model parameters of the initial prediction model are adjusted until the evaluation metric value of the adjusted prediction model meets the preset requirements to obtain the prediction model. The evaluation metric value is calculated from the historical predicted reporting volume input to the prediction model and the historical predicted reporting volume output by the prediction model. The evaluation metric includes at least one of the following: mean squared error and coefficient of determination.
[0043] Specifically, training the predictive model requires multiple sets of sample data, including historical consumption data and historical reporting volumes obtained from the database for a specific time period. The specific time period can be from the initial date to the day before the target time period, where the initial date is the date specified by the financial institution. When training the predictive model using historical consumption data and historical reporting volumes, evaluation metrics are used to determine whether the predictive model has high accuracy.
[0044] Specifically, after inputting historical consumption data into the preset prediction model, multiple historical forecast reports are output. The mean square error E and the coefficient of determination R are then calculated using the historical forecast reports and the historical reports. 2 :
[0045]
[0046]
[0047] Where m represents the number of samples, y i and These represent historical reporting volume and historical predicted reporting volume, respectively. This is the average of historically predicted reporting volumes.
[0048] It should be noted that the preset requirements refer to the calculated mean squared error being sufficiently small and the coefficient of determination being sufficiently large. Mean squared error thresholds and coefficient of determination thresholds can be set. These thresholds are used to determine whether the mean squared error is sufficiently small and the coefficient of determination is sufficiently large, thereby determining whether the evaluation index values meet the preset requirements. If the preset requirements are not met, the parameters of the initial prediction model need to be further adjusted until the prediction model meets the preset requirements. In this embodiment, the evaluation index is used to judge the prediction model during the model training process, enabling the trained prediction model to achieve accurate prediction results.
[0049] To determine the status of the actual reporting volume using statistical methods, optionally, in the data verification method provided in this application embodiment, the actual reporting volume is verified using N predicted reporting volumes to obtain a first verification result. Determining the status of the actual reporting volume based on the first verification result includes: calculating the mean and variance of the N predicted reporting volumes, and using the mean and variance to calculate a test value; comparing the test value with the confidence level, and determining the status of the actual reporting volume as an undetermined state if the test value is greater than or equal to the confidence level, and determining the status of the actual reporting volume as a normal state if the test value is less than the confidence level.
[0050] Specifically, after making multiple predictions of the reporting volume using the prediction model, N predicted reporting volumes are obtained, denoted as a = (a1, a2, a3, ..., a...). n The average value u of the predicted reporting volume is calculated using the following formula:
[0051]
[0052] The variance σ of the predicted reporting volume is calculated using the following formula. 2 And the standard deviation σ:
[0053]
[0054]
[0055] Furthermore, through hypothesis testing methods: Here, b refers to the actual reported quantity. When the confidence level is 95%, or the corresponding confidence level value is 1.96, if z1 ≥ z, it indicates that the actual reported quantity is in an abnormal state; conversely, if z1 < z, it indicates that the actual reported quantity is in a normal state. This embodiment determines the state of the actual reported quantity by using hypothesis testing, thus achieving the effect of accurately determining the state of the reported quantity.
[0056] Optionally, in the data verification method provided in this application embodiment, after determining the status of the actual reported quantity as a pending state when the verification value is greater than or equal to the confidence level, the method further includes: issuing an abnormal prompt message and obtaining the communication information of the inspection personnel; uploading the actual reported quantity to the inspection personnel according to the communication information; receiving the second verification result fed back by the inspection personnel; determining the status of the actual reported quantity as an abnormal state when the first verification result and the second verification result are the same; and determining the status of the actual reported quantity as a normal state when the first verification result and the second verification result are different.
[0057] Specifically, if the hypothesis testing method determines that there is an anomaly in the actual reported quantity, the system obtains the contact information of the inspection personnel and uploads the actual reported quantity to the inspection personnel's terminal based on the contact information. At the same time, it issues an abnormal reported quantity prompt message to notify the inspection personnel to receive and inspect the data in a timely manner.
[0058] Furthermore, after the inspectors perform manual verification, a second verification result is obtained. It is then determined whether this verification result is consistent with the first verification result obtained using the hypothesis testing method. If the two verification results are consistent, it indicates that the actual reported quantity is in an abnormal reporting quantity. Conversely, if the two verification results are inconsistent, that is, the verification result obtained by manual verification is normal, then the reported quantity is determined to be in a normal state. In this embodiment, the reported quantity is uploaded to the inspectors, which can promptly handle abnormal reporting quantities.
[0059] Optionally, in the data verification method provided in this application embodiment, after obtaining the actual reported number of customers whose consumption data generated within the target time period is greater than the consumption data threshold, the method further includes: uploading the actual reported number to the verification personnel when the actual reported number is greater than the preset reported number threshold; and receiving the third verification result fed back by the verification personnel, wherein the third verification result represents the status of the actual reported number.
[0060] Another method for detecting the actual reporting volume is to compare the actual reporting volume with a reporting volume threshold to obtain the status of the actual reporting volume. Specifically, after obtaining the actual reporting volume within the target time period from the database of the financial institution, the reporting volume is compared with a preset reporting volume threshold. When the actual reporting volume is greater than the preset reporting volume threshold, the reporting volume is uploaded for inspection by the inspectors.
[0061] Furthermore, the third verification result fed back by the inspectors after inspection is identified to determine the status of the actual reported quantity, that is, whether it is in a normal or abnormal state.
[0062] Optionally, in the data verification method provided in this application embodiment, obtaining target consumption data generated by multiple customers within a target time period includes: obtaining customers from the blacklist stored by the target institution to obtain multiple customers; and obtaining data generated by multiple customers within the target time period from the database of the target institution to obtain target consumption data.
[0063] Specifically, a blacklist refers to a list of personal information related to illegal transactions or other illegal activities. Target institutions refer to financial institutions that store the blacklist. Multiple customers can include consumers suspected of illegal transactions or other illegal activities. By traversing the financial institution's database based on customer information from the blacklist, the consumption data of the customers to be queried can be obtained and identified as target consumption data. By targeting these customers, the efforts to combat financial crime and the monitoring and management of overseas transactions can be effectively strengthened.
[0064] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.
[0065] This application also provides a data verification system. Figure 2 This is a schematic diagram of a data verification system provided according to an embodiment of this application, such as... Figure 2 As shown, the system includes:
[0066] The system includes a data acquisition module, a reporting volume prediction module, an early warning module, and a reporting data status update module. The data acquisition module is used to acquire target consumption data and actual reporting volume; the reporting volume prediction module predicts the reporting volume using a prediction model; the early warning module provides early warnings when the actual reporting volume is abnormal; and the reporting data status update module updates the status of the actual reporting volume.
[0067] Specifically, after obtaining the target consumption data and actual reporting volume of customers in the blacklist within the target time period from the database of financial institutions, the target consumption data is input into the trained prediction model for multiple predictions to obtain multiple predicted reporting volumes. After verifying the predicted reporting volume and the actual reporting volume using hypothesis testing methods, it is determined whether the status of the actual reporting volume is an abnormal state.
[0068] Furthermore, if the actual reported volume is found to be abnormal after verification, an early warning message is issued, and the actual reported volume is uploaded for manual verification of its status. If the result of the manual verification matches the result obtained through hypothesis testing, it indicates that the actual reported volume is abnormal, and the status of the reported volume is updated to abnormal by updating the reporting data status module. If the result of the manual verification does not match the result obtained through hypothesis testing, it indicates that the actual reported volume is normal, and Li Yang updates the reporting data status module to normal.
[0069] By utilizing the data acquisition module, reporting volume prediction module, early warning module, and reporting data status update module, hypothesis testing methods are used to judge the acquired consumption data and actual reporting volume, and the status of actual reporting volume is obtained in a timely manner, achieving the effect of quickly and accurately identifying the actual reporting volume.
[0070] This application also provides a data verification device. It should be noted that the data verification device of this application can be used to execute the data verification method provided in this application. The data verification device provided in this application is described below.
[0071] Figure 3 This is a schematic diagram of a data verification device provided according to an embodiment of this application, such as... Figure 3 As shown, the device includes: a first acquisition unit 30, a second acquisition unit 31, an input unit 32, and a verification unit 33.
[0072] The first acquisition unit 30 is used to acquire target consumption data generated by multiple customers within a target time period, and to determine the type of the date that generated the target consumption data and the consumption data threshold associated with that type.
[0073] The second acquisition unit 31 is used to acquire the actual reporting volume generated within the target time period, wherein the actual reporting volume refers to the number of customers whose consumption data is greater than the consumption data threshold.
[0074] Input unit 32 is used to input target consumption data into a type-matching prediction model and output N predicted reporting volumes. The prediction model is obtained by training a preset prediction model with M sets of sample data. Each set of sample data includes historical consumption data and historical reporting volumes.
[0075] Verification unit 33 is used to verify the actual reporting volume using N predicted reporting volumes, obtain a first verification result, and determine the status of the actual reporting volume based on the first verification result, wherein the status of the actual reporting volume includes a normal status or an abnormal status.
[0076] Optionally, in the data verification device provided in this application embodiment, the input unit 32 includes: a first input module, used to input target consumption data into a first prediction model when the type is target day, wherein the first prediction model is obtained by training a preset prediction model with M1 sets of first sample data, and the first sample data includes historical target day consumption data and historical target day reporting volume; and a second input module, used to input target consumption data into a second prediction model when the type is non-target day, wherein the second prediction model is obtained by training a preset prediction model with M2 sets of second sample data, and the second sample data includes historical non-target day consumption data and historical non-target day reporting volume, and the sum of M1 and M2 is M.
[0077] Optionally, in the data verification device provided in this application embodiment, the prediction model is trained in the following manner. The training unit includes: a first acquisition module, used to acquire M historical consumption data and historical reporting volume to obtain M sets of sample data; a training module, used to train a preset prediction model using the M sets of sample data to obtain an initial prediction model; and a judgment module, used to judge whether the evaluation index value of the initial prediction model meets the preset requirements, and if the evaluation index value does not meet the preset requirements, to adjust the model parameters of the initial prediction model until the evaluation index value of the adjusted prediction model meets the preset requirements to obtain the prediction model. The evaluation index value is calculated from the historical predicted reporting volume input to the prediction model and the historical predicted reporting volume output by the prediction model. The evaluation index includes at least one of the following: mean square error and coefficient of determination.
[0078] Optionally, in the data verification device provided in this application embodiment, the verification unit 33 includes: a calculation module, used to calculate the mean and variance of N predicted reporting quantities, and use the mean and variance to calculate the test value; and a comparison module, used to compare the test value with the confidence level, and determine the state of the actual reporting quantity as undetermined if the test value is greater than or equal to the confidence level, and determine the state of the actual reporting quantity as normal if the test value is less than the confidence level.
[0079] Optionally, in the data verification device provided in this application embodiment, the device further includes: a sending unit, configured to, when the verification value is greater than or equal to the confidence level, determine the status of the actual reported quantity as a pending state, send an abnormal prompt message, obtain the communication information of the verification personnel, and upload the actual reported quantity to the verification personnel according to the communication information; and a receiving unit, configured to receive the second verification result fed back by the verification personnel, determine the status of the actual reported quantity as an abnormal state when the first verification result and the second verification result are the same, and determine the status of the actual reported quantity as a normal state when the first verification result and the second verification result are different.
[0080] Optionally, in the data verification device provided in this application embodiment, the device further includes: an uploading unit, used to upload the actual reporting volume to the verification personnel when the actual reporting volume is greater than a preset reporting volume threshold after obtaining the actual reporting volume of the number of customers whose consumption data generated within the target time period is greater than the consumption data threshold; and a receiving unit, used to receive the third verification result fed back by the verification personnel, wherein the third verification result represents the status of the actual reporting volume.
[0081] Optionally, in the data verification device provided in this application embodiment, the first acquisition unit 30 includes: a second acquisition module, used to acquire customers in the blacklist stored by the target organization to obtain multiple customers; and a third acquisition module, used to acquire data generated by multiple customers within a target time period from the database of the target organization to obtain target consumption data.
[0082] The data verification device provided in this application embodiment includes a first acquisition unit 30 for acquiring target consumption data generated by multiple customers within a target time period and determining the type of the date that generated the target consumption data and the consumption data threshold associated with that type; a second acquisition unit 31 for acquiring the actual reporting volume generated within the target time period, wherein the actual reporting volume refers to the number of customers whose consumption data exceeds the consumption data threshold; an input unit 32 for inputting the target consumption data into a type-matching prediction model and outputting N predicted reporting volumes, wherein the prediction model is obtained by training a preset prediction model with M sets of sample data, and each set of sample data includes historical consumption data and historical reporting volumes; and a verification unit 33 for verifying the actual reporting volume using the N predicted reporting volumes, obtaining a first verification result, and determining the status of the actual reporting volume based on the first verification result, wherein the status of the actual reporting volume includes a normal status or an abnormal status, solving the problem of difficulty in quickly identifying abnormal reporting volumes in related technologies. By using the prediction model to verify the actual reporting volume and obtaining the verification result, the consumption data is identified using the verification result, thereby achieving the effect of effectively identifying the status of the reporting volume.
[0083] The data verification device includes a processor and a memory. The first acquisition unit 30, the second acquisition unit 31, the input unit 32, the verification unit 33, etc. are all stored in the memory as program units. The processor executes the program units stored in the memory to realize the corresponding functions.
[0084] The processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and adjusting kernel parameters can address the difficulty in quickly identifying abnormal reporting volumes in related technologies.
[0085] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.
[0086] This invention provides a computer-readable storage medium storing a program thereon, which, when executed by a processor, implements the data verification method.
[0087] This invention provides a processor for running a program, wherein the program executes the data verification method during runtime.
[0088] Figure 4 This is a schematic diagram of an electronic device provided according to an embodiment of this application, such as... Figure 4 As shown, this embodiment of the invention provides an electronic device 40, which includes a processor, a memory, and a program stored in the memory and executable on the processor. The processor is used to execute computer-readable instructions, wherein the computer-readable instructions execute a data verification method. The device described herein may be a server, PC, PAD, mobile phone, etc.
[0089] This application also provides a computer program product that, when executed on a data processing device, is suitable for performing a data verification method.
[0090] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0091] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0092] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0093] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0094] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0095] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.
[0096] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.
[0097] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.
[0098] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0099] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.
Claims
1. A data verification method, characterized in that, include: Acquire target consumption data generated by multiple customers within a target time period, and determine the type of the date that generated the target consumption data and the consumption data threshold associated with the type; Obtain the actual reporting volume generated within the target time period, wherein the actual reporting volume refers to the number of customers whose consumption data is greater than the consumption data threshold; The target consumption data is input into the type-matching prediction model, and N predicted reporting volumes are output. The prediction model is obtained by training a preset prediction model with M sets of sample data, and each set of sample data includes historical consumption data and historical reporting volumes. The actual reporting volume is verified using the N predicted reporting volumes to obtain a first verification result. The status of the actual reporting volume is determined based on the first verification result, where the status includes a normal status or an abnormal status. The mean and variance of the N predicted reporting volumes are calculated, and a test value is calculated using the mean and variance. The test value is compared with a confidence level. If the test value is greater than or equal to the confidence level, the status of the actual reporting volume is determined to be an undetermined status; if the test value is less than the confidence level, the status of the actual reporting volume is determined to be a normal status. The test value is calculated using a hypothesis testing method based on the actual reporting volume, the mean, and the variance.
2. The method according to claim 1, characterized in that, The date type that generates the target consumption data includes target days and non-target days. The consumption data threshold associated with the target day is greater than the consumption data threshold associated with the non-target day. Inputting the target consumption data into the type-matching prediction model includes: When the type is the target day, the target consumption data is input into the first prediction model, wherein the first prediction model is obtained by training the preset prediction model with M1 sets of first sample data, and the first sample data includes the consumption data of historical target days and the reporting volume of historical target days. In the case where the type is the non-target day, the target consumption data is input into the second prediction model, wherein the second prediction model is obtained by training the preset prediction model with M2 sets of second sample data, the second sample data including historical non-target day consumption data and historical non-target day reporting volume, and the sum of M1 and M2 is M.
3. The method according to claim 1, characterized in that, The prediction model is trained in the following way: Obtain M historical consumption data points and historical reporting volumes to obtain M sets of sample data; The preset prediction model is trained using the M sets of sample data to obtain the initial prediction model; Determine whether the evaluation index value of the initial prediction model meets the preset requirements. If the evaluation index value does not meet the preset requirements, adjust the model parameters of the initial prediction model until the evaluation index value of the adjusted prediction model meets the preset requirements, thereby obtaining the prediction model. The evaluation index value is calculated from the historical prediction reporting volume input to the prediction model and the historical prediction reporting volume output by the prediction model. The evaluation index includes at least one of the following: mean square error and coefficient of determination.
4. The method according to claim 1, characterized in that, After determining the status of the actual reported quantity as pending when the test value is greater than or equal to the confidence level, the method further includes: Issue an error message and obtain the communication information of the inspection personnel, and upload the actual reported quantity to the inspection personnel based on the communication information; The system receives a second verification result from the inspector. If the first verification result and the second verification result are the same, the system determines that the actual reported quantity is in an abnormal state. If the first verification result and the second verification result are different, the system determines that the actual reported quantity is in a normal state.
5. The method according to claim 1, characterized in that, After obtaining the actual reported number of customers whose consumption data generated within the target time period exceeds the consumption data threshold, the method further includes: If the actual reported amount is greater than the preset reported amount threshold, the actual reported amount will be uploaded to the inspection personnel. The third verification result is received from the inspectors, wherein the third verification result represents the status of the actual reported quantity.
6. The method according to claim 1, characterized in that, Obtaining target consumption data from multiple customers within a target time period includes: Obtain customers from the blacklist stored by the target organization to obtain the multiple customers; The target consumption data is obtained by retrieving data generated by the multiple customers within the target time period from the database of the target institution.
7. A data verification device, characterized in that, include: The first acquisition unit is used to acquire target consumption data generated by multiple customers within a target time period, and to determine the type of the date that generated the target consumption data and the consumption data threshold associated with the type. The second acquisition unit is used to acquire the actual reporting volume generated within the target time period, wherein the actual reporting volume refers to the number of customers whose consumption data is greater than the consumption data threshold. The input unit is used to input the target consumption data into the type-matching prediction model and output N predicted reporting volumes. The prediction model is obtained by training a preset prediction model with M sets of sample data, and each set of sample data includes historical consumption data and historical reporting volumes. A verification unit is configured to verify the actual reporting volume using the N predicted reporting volumes to obtain a first verification result, and determine the status of the actual reporting volume based on the first verification result, wherein the status of the actual reporting volume includes a normal status or an abnormal status; calculate the mean and variance of the N predicted reporting volumes, and calculate a test value using the mean and variance; compare the test value with a confidence level, and if the test value is greater than or equal to the confidence level, determine the status of the actual reporting volume as an undetermined status, and if the test value is less than the confidence level, determine the status of the actual reporting volume as a normal status; the test value is calculated using a hypothesis testing method based on the actual reporting volume, the mean, and the variance.
8. A processor, characterized in that, The processor is used to run a program, wherein the program executes the data verification method according to any one of claims 1 to 6 when it runs.
9. An electronic device, characterized in that, It includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the data verification method according to any one of claims 1 to 6.
Citation Information
Patent Citations
Short-term power load prediction method based on CNN-IPSO-GRU hybrid model
CN111738512A
Method and device for monitoring submitted data fluctuation
CN114896119A