Abnormal data detection method and device, electronic equipment and storage medium

Abstract

The invention discloses an abnormal data detection method and device, electronic equipment and a storage medium, and relates to the technical field of computers.A timestamp of first to-be-detected real-time data is obtained, a pre-training detection model is controlled to generate a predicted value of a corresponding time period according to the timestamp, and when it is determined that a difference value is larger than or equal to a preset threshold value, the abnormal data is detected. Automatically marking the abnormal data and generating alarm information; according to the method, the accuracy and efficiency of data processing are improved, the risk of missing report and false report can be remarkably reduced, so that stable operation and safety of the system are ensured, and the reliability and user satisfaction of the system can be further improved through quick response and processing of abnormal data.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a method and device for detecting abnormal data, an electronic device, and a storage medium. Background Art

[0002] With the digital transformation of the insurance industry and the rapid development of insurance business, insurance data has become increasingly important, and the quality of insurance data has a direct impact on the development of insurance business. However, in actual business, there will be abnormalities in insurance data, and the reasons for the abnormality of insurance data are various, such as: network failure, insurance data format or content errors, system errors, etc. When there are abnormalities in insurance data, it often leads to insurance failure. In order to improve the success rate of insurance, it is necessary to monitor and analyze abnormal data.

[0003] At present, the commonly used abnormal data identification mainly automatically judges abnormalities by setting a large abnormal threshold, such as setting a visit volume threshold based on the historical page visits, and triggering an abnormal alarm when the number of visits exceeds the threshold. Although this method can realize the identification of abnormal data to a certain extent, the method of setting the threshold is relatively rigid and cannot accurately detect abnormal data during the low visit volume period. Summary of the invention

[0004] The present disclosure provides a method, device, electronic device and storage medium for detecting abnormal data, the main purpose of which is to achieve accurate detection of abnormal data.

[0005] According to a first aspect of the present disclosure, a method for detecting abnormal data is provided, comprising:

[0006] Obtaining a timestamp of first real-time data to be detected, and controlling a pre-trained detection model to generate a predicted value for a time period corresponding to the timestamp according to the timestamp and the first real-time data to be detected;

[0007] Determine whether the difference between the first real-time data to be detected and the predicted value is greater than or equal to a preset threshold;

[0008] When it is determined that the difference is greater than or equal to the preset threshold, the first real-time data to be detected is marked, and an alarm message is generated for broadcast.

[0009] Optionally, after determining whether the difference between the first to-be-detected real-time data and the predicted value is greater than or equal to a preset threshold, the method further includes:

[0010] When it is determined that the difference is less than the preset threshold, second real-time data to be detected is obtained, and the second real-time data to be detected is detected according to the pre-trained model.

[0011] Optionally, controlling the generation of a prediction value for a time period corresponding to the timestamp in a pre-trained detection model according to the timestamp includes:

[0012] Retrieving data features corresponding to the time period corresponding to the timestamp in the pre-trained detection model;

[0013] The pre-trained detection model generates a predicted value for the time period corresponding to the timestamp based on the data features and the first real-time data to be detected.

[0014] Optionally, before obtaining the timestamp of the first real-time data to be detected, the method further includes:

[0015] Acquire system data and product data according to preset system detection indicators and preset product detection indicators to obtain the first real-time data to be detected;

[0016] The system data includes at least one of system page views, number of unique visitors, page errors, interface request errors and system performance data;

[0017] The product data includes at least one of product browsing volume, user interaction data, user behavior path, and key product detection.

[0018] Optionally, before controlling the generation of a predicted value for the time period corresponding to the timestamp in the pre-trained detection model according to the timestamp and the first real-time data to be detected, the method further includes:

[0019] Acquire historical data for training, and divide the historical data for training into a training set and a test set according to a preset ratio;

[0020] The pre-trained detection model is trained according to the training set and the test set, and after the number of training times reaches a preset training number threshold, the trained pre-trained detection model is obtained.

[0021] Optionally, after obtaining the trained pre-trained detection model, the method further includes:

[0022] When the product data is updated, the updated product data is obtained, and the pre-trained detection model is trained according to the updated product data.

[0023] According to a second aspect of the present disclosure, there is provided a device for detecting abnormal data, comprising:

[0024] An acquisition unit, used to acquire a timestamp of the first real-time data to be detected, and control the pre-trained detection model to generate a predicted value for the time period corresponding to the timestamp according to the timestamp and the first real-time data to be detected;

[0025] A determination unit, configured to determine whether a difference between the first real-time data to be detected and the predicted value is greater than or equal to a preset threshold;

[0026] The marking unit is used to mark the first real-time data to be detected and generate an alarm message for broadcasting when it is determined that the difference is greater than or equal to the preset threshold.

[0027] Optionally, the device further comprises:

[0028] The first detection unit is used to obtain second real-time data to be detected and detect the second real-time data to be detected according to the pre-trained model after the determination unit determines whether the difference between the first real-time data to be detected and the predicted value is greater than or equal to a preset threshold value, when it is determined that the difference is less than the preset threshold value.

[0029] Optionally, the acquisition unit is further used for:

[0030] Retrieving data features corresponding to the time period corresponding to the timestamp in the pre-trained detection model;

[0031] The pre-trained detection model generates a predicted value for the time period corresponding to the timestamp based on the data features and the first real-time data to be detected.

[0032] Optionally, the device further comprises:

[0033] A second detection unit is used to obtain system data and product data according to preset system detection indicators and preset product detection indicators before the acquisition unit acquires the timestamp of the first real-time data to be detected, so as to obtain the first real-time data to be detected;

[0034] The system data includes at least one of system page views, number of unique visitors, page errors, interface request errors and system performance data;

[0035] The product data includes at least one of product browsing volume, user interaction data, user behavior path, and key product detection.

[0036] Optionally, the device further comprises:

[0037] a dividing unit, configured to obtain historical data for training and divide the historical data for training into a training set and a test set according to a preset ratio before the acquisition unit controls the generation of a predicted value for the time period corresponding to the timestamp in the pre-trained detection model according to the timestamp and the first real-time data to be detected;

[0038] The first training unit is used to train the pre-trained detection model according to the training set and the test set, and obtain the trained pre-trained detection model after the number of training times reaches a preset training number threshold.

[0039] Optionally, the device further comprises:

[0040] The second training unit is used to obtain the updated product data after the pre-trained detection model is trained by the first training unit, when the product data is updated, and train the pre-trained detection model according to the updated product data.

[0041] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0042] at least one processor; and

[0043] a memory communicatively connected to the at least one processor; wherein,

[0044] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect.

[0045] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method described in the first aspect.

[0046] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the method as described in the first aspect above.

[0047] The detection method, device, electronic device and storage medium of abnormal data provided by the present disclosure include: obtaining the timestamp of the first real-time data to be detected, and controlling the pre-trained detection model to generate the predicted value of the time period corresponding to the timestamp according to the timestamp and the first real-time data to be detected; determining whether the difference between the first real-time data to be detected and the predicted value is greater than or equal to the preset threshold. When it is determined that the difference is greater than or equal to the preset threshold, the first real-time data to be detected is marked, and an alarm message is generated for broadcast. Compared with the related art, the embodiment of the present application obtains the timestamp of the first real-time data to be detected, and controls the pre-trained detection model to generate the predicted value of the corresponding time period according to the timestamp, and when it is determined that the difference is greater than or equal to the preset threshold, the abnormal data is automatically marked and an alarm message is generated; not only the accuracy and efficiency of data processing are improved, but also the risk of underreporting and false alarms can be significantly reduced, thereby ensuring the stable operation and security of the system, and the reliability and user satisfaction of the system can be further improved through rapid response and processing of abnormal data.

[0048] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0050] Figure 1 A flowchart of a method for detecting abnormal data provided by an embodiment of the present disclosure;

[0051] Figure 2 A flowchart of a method for detecting abnormal data provided by an embodiment of the present disclosure;

[0052] Figure 3 A schematic diagram of the structure of an abnormal data detection device provided by an embodiment of the present disclosure;

[0053] Figure 4 A schematic diagram of the structure of an abnormal data detection device provided by an embodiment of the present disclosure;

[0054] Figure 5 A schematic block diagram of an exemplary electronic device provided for an embodiment of the present disclosure. DETAILED DESCRIPTION

[0055] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0056] The following describes the abnormal data detection method, device, electronic device and storage medium according to the embodiments of the present disclosure with reference to the accompanying drawings.

[0057] Figure 1 A flowchart of a method for detecting abnormal data provided by an embodiment of the present disclosure.

[0058] like Figure 1 As shown, the method comprises the following steps:

[0059] Step 101, obtain the timestamp of the first real-time data to be detected, and control the pre-trained detection model to generate a predicted value for the time period corresponding to the timestamp according to the timestamp and the first real-time data to be detected.

[0060] In some embodiments, before executing this step, when the first real-time data to be detected is obtained, the timestamp information of the data is immediately extracted; the timestamp is usually a value accurate to seconds or milliseconds, indicating the exact time when the data is generated or received. Specifically, the accuracy of the timestamp can be set according to actual needs, and this application does not limit this. The pre-trained detection model is obtained by training with a large amount of historical data, and can predict the data change trend within a certain period of time for prediction.

[0061] Step 102: determine whether the difference between the first real-time data to be detected and the predicted value is greater than or equal to a preset threshold.

[0062] It is necessary to calculate the difference between the first real-time data to be detected and its corresponding predicted value. Then, we compare this difference with the preset threshold to determine whether there is a significant difference. If the difference is greater than or equal to the preset threshold, it indicates that the current real-time data may be abnormal and further analysis and processing are required. The preset threshold is an empirical value, which can be set according to the standard deviation, such as twice or three times the standard deviation. Specifically, this embodiment of the present application does not limit this.

[0063] Step 103: When it is determined that the difference is greater than or equal to the preset threshold, the first real-time data to be detected is marked, and an alarm message is generated for broadcast.

[0064] When it is determined that the difference is greater than or equal to the preset threshold, the first real-time data to be detected is marked, and an alarm message is generated for broadcasting, so as to promptly remind relevant personnel to pay attention to the abnormal situation and take corresponding measures to deal with it. Specifically, the first real-time data to be detected is marked as an abnormal state for subsequent analysis and tracking, and corresponding alarm messages are generated according to preset alarm rules, and the alarm messages are sent to relevant personnel through various channels, such as SMS, email, instant messaging tools, etc.

[0065] The detection method of abnormal data provided by the present disclosure includes the following main technical solutions: obtaining the timestamp of the first real-time data to be detected, and controlling the pre-trained detection model to generate the predicted value of the time period corresponding to the timestamp according to the timestamp and the first real-time data to be detected; determining whether the difference between the first real-time data to be detected and the predicted value is greater than or equal to the preset threshold. When it is determined that the difference is greater than or equal to the preset threshold, the first real-time data to be detected is marked, and an alarm message is generated for broadcasting. Compared with the related art, the embodiment of the present application obtains the timestamp of the first real-time data to be detected, and controls the pre-trained detection model to generate the predicted value of the corresponding time period according to the timestamp, and when it is determined that the difference is greater than or equal to the preset threshold, the abnormal data is automatically marked and an alarm message is generated; not only the accuracy and efficiency of data processing are improved, but also the risk of underreporting and false alarms can be significantly reduced, thereby ensuring the stable operation and security of the system, and the reliability and user satisfaction of the system can be further improved through rapid response and processing of abnormal data.

[0066] In some embodiments, there is a situation in which the difference in the detection result is less than a preset threshold. The specific steps include: when it is determined that the difference is less than the preset threshold, obtaining second real-time data to be detected, and detecting the second real-time data to be detected according to the pre-trained model.

[0067] After confirming that the difference between the predicted value and the actual value of the first real-time data to be detected is less than a preset threshold, the second real-time data to be detected is obtained and detected, that is, subsequent data points are quickly captured when the conditions are met, and the second real-time data to be detected is detected based on the same pre-trained detection model; continuous data monitoring is achieved to ensure that the system's predictions are consistent with the actual operating conditions, thereby improving the accuracy and efficiency of overall monitoring.

[0068] Figure 2 A flow chart of a method for detecting abnormal data provided by an embodiment of the present disclosure, wherein the control of generating a prediction value for a time period corresponding to the timestamp in a pre-trained detection model according to the timestamp includes:

[0069] Step 201, retrieve the data features corresponding to the time period corresponding to the timestamp in the pre-trained detection model.

[0070] In some embodiments, the pre-trained detection model is obtained by pre-training based on historical data, which includes data features corresponding to each timestamp, such as the data flow size of each time period, flow access features, user information features (IP, access target), statistical attributes of data, trend changes and periodic patterns, etc. Specifically, the embodiments of the present application are not limited to this.

[0071] Step 202: The pre-trained detection model generates a predicted value for the time period corresponding to the timestamp based on the data features and the first real-time data to be detected.

[0072] The pre-trained detection model first integrates these two pieces of information to ensure the effective combination of real-time data and historical features. Then, the model analyzes and processes these comprehensive features through its internal algorithms and parameters, and uses the learned patterns and capabilities to predict data trends or the possibility of events occurring within that time period.

[0073] Optionally, before obtaining the timestamp of the first real-time data to be detected, the method further includes:

[0074] In some embodiments, before obtaining the real-time data to be detected, it is first necessary to collect data according to a preset data collection method, obtain system data and product data according to preset system detection indicators and preset product detection indicators, and obtain the first real-time data to be detected;

[0075] The system data includes at least one of system page views, number of unique visitors, page errors, interface request errors and system performance data;

[0076] The product data includes at least one of product browsing volume, user interaction data, user behavior path, and key product detection.

[0077] System-oriented monitoring indicators include:

[0078] System visits: PV (Page View), that is, page views, refers to the total number of times the system page is visited. Every time a user loads or reloads a page, PV increases once; UV (Unique Visitor), that is, the number of unique visitors, refers to the number of unique users who visit the system within a certain period of time. Even if a user visits the system multiple times in a day, UV is only counted once; through the system PV and UV values, the stability of the system operation can be monitored, and crawler attacks can be discovered and avoided in advance.

[0079] System errors / exceptions: JavaScript errors: capture errors that occur when the insurance process page is running, including syntax errors triggered at runtime, custom errors, interface errors, error times, error messages, error rates, error types, error distribution devices (Android / IOS / PC), etc.; Interface request errors: record the interface requests in the insurance process, including interface name, number of requests, number of errors, error rate, request URL, request duration, HTTP status code, etc.; System performance data: average DOM parsing time of the page, average page loading time, average TTFB (Time To First Byte) time, average interface request time, average resource loading time, etc.

[0080] Product-oriented monitoring indicators include:

[0081] Page browsing data: record the URL of the page that users visit for a certain product (home screen, information confirmation page, signature page, payment page, etc.) and the time they stay on the page, and learn the normal user behavior patterns. If the page stay time is too short during the insurance process, it may be that the front-end verification is bypassed; if the stay time is too long and involves many users, it may be that the page performance is abnormal.

[0082] User interaction data: Click events: record the buttons and links clicked by users, focusing on several key nodes in the insurance process, such as the click frequency of buttons such as "Insure Now" and "Confirm Payment". If the click frequency is too fast, it may be a malicious interface attack; scrolling and touch events: the depth and frequency of page scrolling, such as the mandatory insurance terms and special agreements that users browse when they apply for insurance. Touch operations of mobile users, such as handwritten signatures on the signature page in the insurance process. Through scrolling events and touch events, learn the normal user behavior patterns, analyze the page fluency, and help developers improve the user's insurance experience in a targeted manner; user behavior path, record the user's browsing and operation path on the insurance page, and help developers quickly troubleshoot errors / abnormalities; special product monitoring, for non-fixed fixed-rate products, monitor the premium of the policy. For example, under normal circumstances, there should be a certain distribution ratio according to different plans, whether there is social insurance, and the age of the insured. If the premium of the policy is relatively simple, it should be checked whether there is a problem with the calculation of the fee or the case of brushing.

[0083] In some embodiments, for a multi-plan product, the click frequency of the plan switching button is monitored. If it is too fast, it may be that the plan display is abnormal or has been attacked.

[0084] For key products, during peak periods, interfaces of key nodes including but not limited to billing, verification (duplicate verification, critical illness verification, risk control verification, etc.), generation of insurance policies, and payment are monitored. Specific monitoring indicators vary according to product characteristics, and abnormal situations of interfaces in complex production environments are discovered in advance. If the response time becomes longer or the request error rate increases, it may be a scenario that cannot be reproduced during stress testing, and attention should be paid to ensure the robustness of the system.

[0085] In some embodiments, the above-mentioned system-oriented and product-oriented monitoring indicators can be supplemented and modified as the system is iterated and the product is updated.

[0086] In some embodiments, before controlling the generation of a predicted value for the time period corresponding to the timestamp in the pre-trained detection model according to the timestamp and the first real-time data to be detected, the method further includes:

[0087] Acquire historical data for training, and divide the historical data for training into a training set and a test set according to a preset ratio;

[0088] The pre-trained detection model is trained according to the training set and the test set, and after the number of training times reaches a preset training number threshold, the trained pre-trained detection model is obtained.

[0089] LSTM introduces "memory units" (cell states) and gating mechanisms (gates), which enable the network to autonomously learn when to "remember" information, when to "forget" information, and when to "read" information, thereby effectively processing long-term dependencies in sequence data. This model has achieved remarkable success in the field of time series analysis, so this neural network model was selected to perform time series analysis on the embedded point data of the front-end page of the insurance process.

[0090] In the LSTM model, the training set data is taken as 1 hour value, the size of the input history sequence window is set to T = 24 (i.e. the time span is 1 day), the number of hidden layer nodes is initially set to 40, the learning rate is 0.0001, the batch size is 256, and a total of 120 epochs are trained. Based on the error or residual predicted by the model, the threshold of anomaly detection is set to the principle of three times the standard deviation. According to the actual training effect, the above model parameter values ​​can be further adjusted to improve the model capability.

[0091] The real-time monitoring data of the front-end points of the insurance process is input into the model to calculate the deviation between the predicted value and the actual value, and those above the threshold are marked as abnormal. At the same time, an alarm system is established to count and sort out the corresponding solutions for common and representative problems. When the alarm is immediately notified to specific personnel through corporate WeChat / email, it also prompts how to handle / troubleshoot such errors, so as to quickly respond to abnormal behaviors detected by the model and promptly discover and solve system failures that may lead to a decline in user experience.

[0092] In some embodiments, after obtaining the trained pre-trained detection model, it also includes: when the product data is updated, obtaining the updated product data, and training the pre-trained detection model according to the updated product data.

[0093] In some embodiments, model parameters can be adjusted based on daily usage feedback, feature value selection can be supplemented or modified based on product updates, and model monitoring accuracy can be continuously optimized.

[0094] Corresponding to the above-mentioned abnormal data detection method, the present invention also provides an abnormal data detection device. Since the device embodiment of the present invention corresponds to the above-mentioned method embodiment, details not disclosed in the device embodiment can be referred to the above-mentioned method embodiment, and will not be repeated in the present invention.

[0095] Figure 3 A schematic diagram of a structure of an abnormal data detection device provided by an embodiment of the present disclosure, such as Figure 3 As shown, including:

[0096] An acquisition unit 31 is used to acquire a timestamp of the first real-time data to be detected, and control the pre-trained detection model to generate a predicted value of the time period corresponding to the timestamp according to the timestamp and the first real-time data to be detected;

[0097] A determination unit 32, configured to determine whether a difference between the first real-time data to be detected and the predicted value is greater than or equal to a preset threshold;

[0098] The marking unit 33 is used to mark the first real-time data to be detected and generate alarm information for broadcast when it is determined that the difference is greater than or equal to the preset threshold.

[0099] The detection device of abnormal data provided by the present disclosure has the following main technical solutions: obtaining the timestamp of the first real-time data to be detected, and controlling the pre-trained detection model to generate the predicted value of the time period corresponding to the timestamp according to the timestamp and the first real-time data to be detected; determining whether the difference between the first real-time data to be detected and the predicted value is greater than or equal to the preset threshold. When it is determined that the difference is greater than or equal to the preset threshold, the first real-time data to be detected is marked, and an alarm message is generated for broadcasting. Compared with the related art, the embodiment of the present application obtains the timestamp of the first real-time data to be detected, and controls the pre-trained detection model to generate the predicted value of the corresponding time period according to the timestamp, and when it is determined that the difference is greater than or equal to the preset threshold, the abnormal data is automatically marked and an alarm message is generated; not only the accuracy and efficiency of data processing are improved, but also the risk of underreporting and false alarms can be significantly reduced, thereby ensuring the stable operation and security of the system, and the reliability and user satisfaction of the system can be further improved through rapid response and processing of abnormal data.

[0100] Furthermore, in a possible implementation of the embodiment of the present disclosure, as Figure 4 As shown, the device also includes:

[0101] The first detection unit 34 is used to obtain the second real-time data to be detected and detect the second real-time data to be detected according to the pre-trained model after the determination unit 32 determines whether the difference between the first real-time data to be detected and the predicted value is greater than or equal to a preset threshold, when it is determined that the difference is less than the preset threshold.

[0102] Furthermore, in a possible implementation of the embodiment of the present disclosure, the acquisition unit 31 is further configured to:

[0103] Retrieving data features corresponding to the time period corresponding to the timestamp in the pre-trained detection model;

[0104] The pre-trained detection model generates a predicted value for the time period corresponding to the timestamp based on the data features and the first real-time data to be detected.

[0105] Furthermore, in a possible implementation of the embodiment of the present disclosure, as Figure 4 As shown, the device also includes:

[0106] The second detection unit 35 is used to obtain system data and product data according to preset system detection indicators and preset product detection indicators before the acquisition unit 31 acquires the timestamp of the first real-time data to be detected, so as to obtain the first real-time data to be detected;

[0107] The system data includes at least one of system page views, number of unique visitors, page errors, interface request errors and system performance data;

[0108] The product data includes at least one of product browsing volume, user interaction data, user behavior path, and key product detection.

[0109] Furthermore, in a possible implementation of the embodiment of the present disclosure, as Figure 4 As shown, the device also includes:

[0110] A division unit 36 ​​is used to obtain training historical data before the acquisition unit 31 controls the generation of a predicted value for the time period corresponding to the timestamp in the pre-trained detection model according to the timestamp and the first real-time data to be detected, and divide the training historical data into a training set and a test set according to a preset ratio;

[0111] The first training unit 37 is used to train the pre-trained detection model according to the training set and the test set, and obtain the trained pre-trained detection model after the number of training times reaches a preset training number threshold.

[0112] Furthermore, in a possible implementation of the embodiment of the present disclosure, as Figure 4 As shown, the device also includes:

[0113] The second training unit 38 is used to obtain the updated product data after the pre-trained detection model is trained by the first training unit 37, and train the pre-trained detection model according to the updated product data when the product data is updated.

[0114] It should be noted that the above explanation of the method embodiment is also applicable to the device of the embodiment of the present disclosure, and the principle is the same, which is no longer limited in the embodiment of the present disclosure.

[0115] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0116] Figure 5A schematic block diagram of an example electronic device 400 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0117] like Figure 5 As shown, the device 400 includes a computing unit 401, which can perform various appropriate actions and processes according to a computer program stored in a ROM (Read-Only Memory) 402 or a computer program loaded from a storage unit 408 to a RAM (Random Access Memory) 403. In the RAM 403, various programs and data required for the operation of the device 400 can also be stored. The computing unit 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An I / O (Input / Output) interface 405 is also connected to the bus 404.

[0118] A number of components in the device 400 are connected to the I / O interface 405, including: an input unit 406, such as a keyboard, a mouse, etc.; an output unit 407, such as various types of displays, speakers, etc.; a storage unit 408, such as a disk, an optical disk, etc.; and a communication unit 409, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 409 allows the device 400 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0119] The computing unit 401 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, CPU (Central Processing Unit), GPU (Graphic Processing Units), various dedicated AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, DSP (Digital Signal Processor), and any appropriate processor, controller, microcontroller, etc. The computing unit 401 performs the various methods and processes described above, such as a method for detecting abnormal data. For example, in some embodiments, the method for detecting abnormal data may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 408. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 400 via ROM 402 and / or communication unit 409. When the computer program is loaded into RAM 403 and executed by the computing unit 401, one or more steps of the method described above may be performed. Alternatively, in other embodiments, the computing unit 401 may be configured to execute the aforementioned abnormal data detection method in any other appropriate manner (for example, by means of firmware).

[0120] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application Specific Standard Products), SOCs (System On Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that may be executed and / or interpreted on a programmable system including at least one programmable processor that may be a special purpose or general purpose programmable processor that may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0121] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0122] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a RAM, a ROM, an EPROM (Electrically Programmable Read-Only-Memory) or a flash memory, an optical fiber, a CD-ROM (Compact Dis sc Read-Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0123] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball), through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0124] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: LAN (Local Area Network), WAN (Wide Area Network), the Internet, and blockchain networks.

[0125] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0126] It should be noted that artificial intelligence is a discipline that studies how computers can simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.), and includes both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, as well as machine learning / deep learning, big data processing technology, knowledge graph technology, and other major directions.

[0127] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0128] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for detecting abnormal data, characterized in that: include: Obtaining a timestamp of first real-time data to be detected, and controlling a pre-trained detection model to generate a predicted value for a time period corresponding to the timestamp according to the timestamp and the first real-time data to be detected; Determine whether the difference between the first real-time data to be detected and the predicted value is greater than or equal to a preset threshold; When it is determined that the difference is greater than or equal to the preset threshold, the first real-time data to be detected is marked, and an alarm message is generated for broadcast.

2. The method according to claim 1, characterized in that After determining whether the difference between the first to-be-detected real-time data and the predicted value is greater than or equal to a preset threshold, the method further includes: When it is determined that the difference is less than the preset threshold, second real-time data to be detected is obtained, and the second real-time data to be detected is detected according to the pre-trained model.

3. The method according to claim 1, characterized in that: The step of controlling the generation of a prediction value for a time period corresponding to the timestamp in a pre-trained detection model according to the timestamp includes: Retrieving data features corresponding to the time period corresponding to the timestamp in the pre-trained detection model; The pre-trained detection model generates a predicted value for the time period corresponding to the timestamp based on the data features and the first real-time data to be detected.

4. The method according to claim 1, characterized in that: Before obtaining the timestamp of the first real-time data to be detected, the method further includes: Acquire system data and product data according to preset system detection indicators and preset product detection indicators to obtain the first real-time data to be detected; The system data includes at least one of system page views, number of unique visitors, page errors, interface request errors and system performance data; The product data includes at least one of product browsing volume, user interaction data, user behavior path, and key product detection.

5. The method according to claim 4, characterized in that Before controlling the generation of a predicted value for the time period corresponding to the timestamp in the pre-trained detection model according to the timestamp and the first real-time data to be detected, the method further includes: Acquire historical data for training, and divide the historical data for training into a training set and a test set according to a preset ratio; The pre-trained detection model is trained according to the training set and the test set, and after the number of training times reaches a preset training number threshold, the trained pre-trained detection model is obtained.

6. The method according to claim 5, characterized in that After obtaining the trained pre-trained detection model, the method further includes: When the product data is updated, the updated product data is obtained, and the pre-trained detection model is trained according to the updated product data.

7. A device for detecting abnormal data, characterized in that: include: An acquisition unit, used to acquire a timestamp of the first real-time data to be detected, and control the pre-trained detection model to generate a predicted value for the time period corresponding to the timestamp according to the timestamp and the first real-time data to be detected; A determination unit, configured to determine whether a difference between the first real-time data to be detected and the predicted value is greater than or equal to a preset threshold; The marking unit is used to mark the first real-time data to be detected and generate an alarm message for broadcasting when it is determined that the difference is greater than or equal to the preset threshold.

8. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-6.

10. A computer program product, characterized in that The invention comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images