Abnormal data detection method, device, electronic device and storage medium

Abstract

The present application provides an abnormal data detection method, device, electronic device and storage medium. The method includes: obtaining historical reservation data, real-time reservation data and abnormal pattern labels; training a detection model based on the historical reservation data and the abnormal pattern labels; detecting the real-time reservation data according to the detection model to determine the abnormality of the real-time reservation data; determining whether the abnormality is higher than a preset first abnormality threshold; in response to the abnormality being higher than the preset first abnormality threshold, determining that the real-time reservation data is abnormal data. Based on the historical reservation data, information of multiple dimensions is aggregated, and the real-time reservation data belonging to the abnormal pattern is determined before the user generates any behavioral data, and then the account belonging to the abnormal data is determined, thereby improving the accuracy and coverage of abnormal identification, significantly reducing the abnormal identification range of the original reservation data, reducing labor costs, improving detection efficiency, and ensuring the consistency of detection results.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an abnormal data detection method, device, electronic device and storage medium. Background Art

[0002] Among the related technologies, the focus is mainly on offline abnormal data identification, which performs anomaly detection on existing accounts and directly applies it to online reasoning. Based on the roles actually created by the users, the data that generates the behavior sequence is determined, and then the abnormal situation of the account is judged through offline analysis of user portraits.

[0003] However, the relevant technology needs to obtain data on the behavior sequence that has been generated, and therefore cannot solve the scenario requirements of abnormal data judgment in the new game server reservation scenario, and it is difficult to ensure the accuracy and consistency of the detection results. Summary of the invention

[0004] In view of this, the purpose of the present application is to provide an abnormal data detection method, device, electronic device and storage medium.

[0005] Based on the above objectives, in a first aspect, the present application provides a method for detecting abnormal data, comprising:

[0006] Get historical reservation data, real-time reservation data, and abnormal pattern labels;

[0007] training a detection model based on the historical reservation data and abnormal pattern labels;

[0008] Detecting the real-time reservation data according to the detection model to determine the abnormality of the real-time reservation data;

[0009] Determining whether the abnormality is higher than a preset first abnormality threshold;

[0010] In response to the abnormality being higher than a preset first abnormality threshold, the real-time reservation data is determined to be abnormal data.

[0011] In a possible implementation, the acquiring of historical reservation data includes:

[0012] Get user data;

[0013] Determining account data and historical user behavior data associated with the user data based on the user data;

[0014] Determine the network data and hard disk data bound to the account data according to the account data;

[0015] The historical reservation data is determined based on the user data, account data, historical user behavior data, network data and hard disk data.

[0016] In a possible implementation, the training of the detection model according to the historical reservation data and the abnormal pattern label includes:

[0017] Determine a plurality of target training data and user historical portraits according to the historical reservation data;

[0018] Determine the appointment time according to the historical appointment data;

[0019] Determining continuity information of the plurality of target training data according to the reservation time;

[0020] Encoding the plurality of target training data according to a preset encoding rule to determine a plurality of target training data encodings;

[0021] Associating the plurality of target training data codes, user history profiles and continuity information to determine training samples;

[0022] The detection model is trained according to the training samples and the abnormal pattern labels.

[0023] In a possible implementation, determining the continuity information of the plurality of target training data according to the reservation time includes:

[0024] Sequentially sorting the plurality of target training data according to the appointment time;

[0025] Determine whether the time interval between adjacent target training data is less than a preset time interval according to the reservation time;

[0026] In response to a time interval between the adjacent target training data being less than the preset time interval, determining that the adjacent target training data are continuous;

[0027] In response to the time interval between the adjacent target training data being not less than the preset time interval, it is determined that the adjacent target training data are not continuous.

[0028] In a possible implementation manner, encoding the plurality of target training data according to a preset encoding rule to determine the plurality of target training data encodings includes:

[0029] Classifying the plurality of target training data to determine at least two character types; wherein each character type corresponds to a different preset code;

[0030] sequentially determining the character type corresponding to each character in the plurality of target training data, and the statistical number of consecutive appearances of the character type;

[0031] Each character in the plurality of target training data is encoded according to the preset encoding and the statistical quantity to determine a plurality of target training data encodings.

[0032] In a possible implementation, the step of training the detection model according to the training sample and the abnormal pattern label includes:

[0033] Determine an intra-class vector corresponding to each target training data encoding;

[0034] Select any one of the target training data codes to be determined as a target screening condition, determine the intra-class distance between each of the remaining target training data codes and the target screening condition according to the intra-class vector, and obtain the intra-class distance corresponding to the target screening condition;

[0035] The target screening condition is updated to any one of the target training data codes that have not been selected, and the step of determining the intra-class distance between each of the remaining target training data codes and the target screening condition is repeated until the intra-class distance corresponding to each of the target training data codes is obtained;

[0036] Determining abnormal data according to the abnormal pattern label;

[0037] Determine an abnormal intra-class distance according to the abnormal data and all the intra-class distances;

[0038] The detection model is trained based on the intra-anomaly distance.

[0039] In a possible implementation, the real-time reservation data includes: a timestamp;

[0040] The detecting the real-time reservation data according to the detection model to determine the abnormality of the real-time reservation data includes:

[0041] storing the real-time reservation data according to the time sequence of the timestamps through a distributed processing engine, and marking the real-time reservation data as unreasoned data;

[0042] Performing hash mapping on the unreasoned data to determine a first hash value corresponding to the unreasoned data;

[0043] Acquire multiple reasoning instances; wherein each of the reasoning instances includes a second Hash value;

[0044] Matching the first hash value and the second hash value to assign the un-inferred data to the inference instance;

[0045] Synchronize all the inference instances, and read the real-time reservation data of N time windows before the timestamp according to the timestamp of the un-inferred data to determine the data to be detected, wherein N is a positive integer;

[0046] The data to be detected is detected according to the detection model to determine the abnormality of the data to be detected.

[0047] In a possible implementation, the method further includes:

[0048] Set the recovery time node;

[0049] In response to the detection model stopping service, resetting the real-time reservation data within the restoration time node and the current time to the un-reasoned data according to the restoration time node;

[0050] Clearing the abnormality degree and abnormal data of the real-time reservation data between the restoration time node and the current time;

[0051] The detection model is restarted according to the recovery time node.

[0052] In a possible implementation manner, after detecting the real-time reservation data according to the detection model to determine the abnormality of the real-time reservation data, the method further includes:

[0053] In response to the abnormality being higher than the preset second abnormality threshold, performing cluster analysis on the real-time reservation data;

[0054] Determine a new abnormal pattern label according to the real-time reservation data corresponding to the abnormality degree higher than the second abnormality threshold;

[0055] The detection model is updated according to the real-time reservation data and the new abnormal pattern label based on online learning technology.

[0056] In a possible implementation, the second abnormality threshold is greater than the first abnormality threshold.

[0057] In a second aspect, the present application provides an abnormal data detection device, comprising:

[0058] an acquisition module configured to acquire historical reservation data, real-time reservation data, and abnormal pattern labels;

[0059] A training module, configured to train a detection model based on the historical reservation data and abnormal pattern labels;

[0060] A first determination module is configured to detect the real-time reservation data according to the detection model to determine the abnormality of the real-time reservation data;

[0061] A second determination module is configured to determine whether the abnormality is higher than a preset first abnormality threshold;

[0062] The detection module is configured to determine that the real-time reservation data is abnormal data in response to the abnormality being higher than the preset first abnormality threshold.

[0063] In a third aspect, the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the abnormal data detection method as described in the first aspect is implemented.

[0064] In a fourth aspect, the present application provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, wherein the computer instructions are used to enable a computer to execute the abnormal data detection method as described in the first aspect.

[0065] From the above, it can be seen that the abnormal data detection method, device, electronic device and storage medium provided by the present application can obtain historical reservation data, real-time reservation data and abnormal pattern labels in the scenario of new game server reservation, and train the detection model through historical reservation data and abnormal pattern labels, so that the real-time reservation data can be detected through the detection model before the user creates a character and generates behavioral data, and the abnormal degree of the obtained real-time reservation data is compared with the preset first abnormal degree threshold to further determine whether the real-time reservation data is abnormal data. According to the historical reservation data, information of multiple dimensions is aggregated, and the real-time reservation data belonging to the abnormal pattern is determined before the user generates any behavioral data, and then the account belonging to the abnormal data is determined, thereby improving the accuracy and coverage of abnormal identification, significantly reducing the abnormal identification range of the original reservation data, reducing labor costs, improving detection efficiency, and ensuring the consistency of the detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the technical solutions in the present application or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0067] Figure 1 An exemplary flow chart of an abnormal data detection method provided in an embodiment of the present application is shown.

[0068] Figure 2 An exemplary flow chart of a method for determining a user history portrait according to an embodiment of the present application is shown.

[0069] Figure 3 An exemplary flowchart of a method for determining an abnormality degree according to an embodiment of the present application is shown.

[0070] Figure 4 An exemplary flowchart of abnormal data detection combined with online learning according to an embodiment of the present application is shown.

[0071] Figure 5 A schematic diagram of an exemplary structure of an abnormal data detection device provided in an embodiment of the present application is shown.

[0072] Figure 6 A schematic diagram of an exemplary structure of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0073] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0074] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be the usual meanings understood by people with ordinary skills in the field to which the present application belongs. The "first", "second" and similar words used in the embodiments of the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0075] Taking the reservation scenario of opening a new game server as an example, after the reservation for a new game server is opened, it usually lasts for a period of time, such as a week. In addition to the reservation behavior of normal players, third-party teams will also make large-scale reservations through technical means, and trigger subsequent violations of game rules. For example, after the reservation is successful, entering the game and hanging up affects the user experience of normal players, transferring game currency through multiple small accounts, and destroying the value system of the game ecology.

[0076] Generally speaking, game developers can use key information such as mobile phones, email accounts, MAC addresses, etc. to associate game characters with real players, and use statistical rules, machine learning and other methods to analyze the abnormality of game accounts and take further measures to avoid possible abnormal behavior.

[0077] As described in the background technology section, the relevant technologies mainly focus on offline abnormal data identification, perform anomaly detection on existing accounts, and directly apply it to online reasoning. According to the role actually created by the user, the data that generates the behavior sequence is determined, and then the abnormal situation of the account is judged through offline analysis of user portraits.

[0078] However, the relevant technology needs to obtain data on the behavior sequence that has been generated, and therefore cannot solve the scenario requirements of abnormal data judgment in the new game server reservation scenario, and it is difficult to ensure the accuracy and consistency of the detection results.

[0079] At the same time, the applicant discovered through research that the existing technical solutions use offline data training models or methods to identify abnormal accounts and directly use them for online reasoning. In practical applications, existing abnormal accounts and new accounts that meet the abnormal pattern can be identified, but they cannot solve the scenario requirements of new game server reservations. For example, a large number of reservation accounts will pour in within a short period of time, and a single instance cannot guarantee good real-time performance during reasoning, or the detection service is unavailable, the detection effect needs to be iterated, and the service needs to be suspended and restarted. In this case, it is difficult to ensure the consistency and reproducibility of the detection results.

[0080] For this reason, the present application provides an abnormal data detection method, device, electronic device and storage medium. In the scenario of new game server reservation, historical reservation data, real-time reservation data and abnormal pattern labels can be obtained, and the detection model can be trained through historical reservation data and abnormal pattern labels, so that the real-time reservation data can be detected through the detection model before the user creates a character and generates behavioral data. By comparing the abnormality of the obtained real-time reservation data with the preset first abnormality threshold, it is further determined whether the real-time reservation data is abnormal data. Based on the historical reservation data, information of multiple dimensions is aggregated, and the real-time reservation data belonging to the abnormal pattern is determined before the user generates any behavioral data, and then the account belonging to the abnormal data is determined, thereby improving the accuracy and coverage of abnormal identification, significantly reducing the abnormal identification range of the original reservation data, reducing labor costs, improving detection efficiency, and ensuring the consistency of detection results.

[0081] The abnormal data detection method provided in the embodiment of the present application is described in detail below through specific examples.

[0082] Figure 1 An exemplary flow chart of an abnormal data detection method provided in an embodiment of the present application is shown.

[0083] refer to Figure 1 The abnormal data detection method provided in the embodiment of the present application specifically includes the following steps:

[0084] S102: Acquire historical reservation data, real-time reservation data and abnormal pattern labels.

[0085] S104: Training a detection model according to the historical reservation data and abnormal pattern labels.

[0086] S106: Detect the real-time reservation data according to the detection model to determine the abnormality of the real-time reservation data.

[0087] S108: Determine whether the abnormality is higher than a preset first abnormality threshold.

[0088] S110: In response to the abnormality being higher than a preset first abnormality threshold, determining that the real-time reservation data is abnormal data.

[0089] For step S102, taking the application scenario of the game new server opening reservation stage as an example, the game new server reservation refers to the reservation service provided when adding new servers or opening the service after the game is online for a period of time, or because of the operation of the game party, etc., the new server reservation can reduce the queuing problem of users when opening the service and reasonably plan the traffic. However, due to the existence of a large number of reservations by third-party teams through technical means, subsequent violations of game rules are triggered, such as after the reservation is successful, entering the game after the service is opened and other behaviors affect the user experience of normal users, or transferring game currency through multiple reservation accounts, destroying the value system of the game ecology, etc. At this time, before the user officially enters the game, in order to ensure the good operation of the game environment and the good experience of the user, the user who makes the reservation can be detected, and users with abnormal behavior or existing abnormal modes can be found in advance, and the accounts to which the abnormal data belongs can be screened out, and processed in advance. This application obtains historical reservation data and existing abnormal modes, converts the existing abnormal modes into abnormal mode tags, and screens users through abnormal mode tags and accounts and user data associated with historical reservation data.

[0090] In some embodiments, the game party can associate game characters and real users through key information, such as mobile phone numbers, email accounts, MAC addresses, etc., and therefore obtain historical reservation data by obtaining user data, such as the user's mobile phone number and user email account provided to the game party when making a reservation, and further determining the account data and historical user behavior data associated with the user data based on the user data, where the account data can be data such as the user's game account, game nickname, number of game characters, game online time, etc. in games that the user has played, and the historical user behavior data can be data recorded on whether the user has abnormal behavior in games that have been played.

[0091] Furthermore, the network data and hard disk data bound to the account data can be determined based on the account data, wherein the network data can include the IP location and data set corresponding to the service provider to which the user corresponding to the account data belongs when booking a game or entering a game in the historical records. For example, based on the account data of a certain user, the IP address and / or MAC address to which the user belongs when booking other online games can be mapped to "X country|X province|X city|X district|Telecom". The hard disk data can include the hard disk manufacturer, hard disk type, hard disk capacity and serial number, such as "DerlerSSD240G-DLDS1721001090", where "Derler" represents the hard disk manufacturer, "SSD" represents the hard disk type, "240G" represents the hard disk capacity, and "DLDS1721001090" represents the serial number.

[0092] Furthermore, data sets of different users may be established based on user data, account data, historical user behavior data, network data, and hard disk data, and the historical reservation data of different users may be determined.

[0093] With respect to step S104, the user's historical reservation data and abnormal pattern labels have been obtained in step S102, and the abnormal pattern labels can be used as training labels and added to the user's historical reservation data to train the detection model.

[0094] Figure 2 An exemplary flow chart of a method for determining a user history portrait according to an embodiment of the present application is shown.

[0095] In some embodiments, multiple target training data and user historical profiles can be determined based on historical reservation data. Figure 2 Specifically, a game data set can be obtained, including data of several existing games, and then associated based on a unique keyword, such as a user's mobile phone number or a user's email account in the user data. Taking the user's mobile phone number as an example, the user's mobile phone number corresponds to a user, and the game data set is traversed based on the user's mobile phone number to obtain the account data and historical user behavior data of all servers in all games associated with the user's mobile phone number, and then the user's historical profile of the user is determined based on all the account data and historical user behavior data associated with the user's mobile phone number.

[0096] For target training data, different data can be selected from the historical reservation data as target training data according to the specific implementation scenario. The subsequent embodiments take the selection of MAC address, hard disk data, email account prefix and game nickname as target training data as an example.

[0097] Since third-party studios often have temporal continuity when they make large-scale reservations for game accounts, we can determine the reservation time of these data based on historical reservation data, determine the continuity information of the target training data based on the reservation time, and filter out the accounts that are batch-reserved by the studio in the historical reservation data. We can use abnormal pattern labels to mark this type of historical reservation data.

[0098] Figure 3 An exemplary flowchart of a method for determining an abnormality degree according to an embodiment of the present application is shown.

[0099] refer to Figure 3 Furthermore, multiple target training data can be sorted in chronological order according to the appointment time, a preset time interval T can be set, and it can be determined whether the time interval between adjacent target training data is less than the preset time interval T. If the time interval between adjacent target training data is less than the preset time interval T, it proves that there is continuity between adjacent target training data and can be marked as 1; if the time interval between adjacent target training data is not less than the preset time interval T, it proves that there is no continuity between adjacent target training data and can be marked as 0.

[0100] Furthermore, because different target training data are represented in different ways, for example, hard disk data is represented by a large number of character strings of numbers and letters, but game nicknames are generally represented by a large number of Chinese character strings. In order to enable each target training data to be represented by a unified standard and reduce the system storage pressure at the same time, multiple target training data can be encoded according to preset encoding rules, so that the target training data is represented by the target training data encoding.

[0101] In some embodiments, multiple target training data may be classified to determine at least two character types, each character type corresponding to a different preset code, and the character type corresponding to each character in the multiple target training data is further determined in sequence, and the number of consecutive appearances of the character type in each target training data is counted, and each character in the multiple target training data is encoded according to the preset code corresponding to the character type and the counted number, thereby determining the encoding of the multiple target training data. This is described below through a specific embodiment.

[0102] For MAC addresses and email account prefixes, Chinese characters, numbers, letters, and other characters may appear. Among them, numbers and letters can be uniformly encoded. For example, the preset encoding corresponding to the Chinese character type can be c, the unified preset encoding corresponding to the number and letter character type can be an, and the preset encoding corresponding to the other character type can be o. Further, count the statistical quantity of each character type continuously appearing in the target training data, and encode the target training data according to the preset encoding and the statistical quantity. For example, a target training data is "拥G赵正琛Ck4%" (Yong G Zhao Zhengchen Ck4%). "拥" (Yong) is encoded as c, "G" is encoded as an, each character in "赵正琛" (Zhao Zhengchen) is encoded as c, each character in "Ck4" is encoded as an, "%" is encoded as o. Count the statistical quantity of each character type continuously appearing, then the target training data "拥G赵正琛Ck4%" is encoded as "1c1an3c3an1o".

[0103] For game nicknames, they generally include Chinese characters, numbers, letters, and other symbols. For example, the preset encoding corresponding to the Chinese character type can be c, the preset encoding corresponding to the number character type is d, the preset encoding corresponding to the letter character type can be a, and the preset encoding corresponding to the other character type can be o. Further, count the statistical quantity of each character type continuously appearing in the target training data, and encode the target training data according to the preset encoding and the statistical quantity. For example, a target training data is "豹子头0充T1!" (Baotou 0 Chong T1!). Each character in "豹子头" (Baotou) is encoded as c, "0" is encoded as d, "充" (Chong) is encoded as c, "T" is encoded as a, "1" is encoded as d, "!" is encoded as o. Count the statistical quantity of each character type continuously appearing, then the target training data "豹子头0充T1!" is encoded as "3c1d1c1a1d1o".

[0104] For hard disk data, in addition to Chinese characters, numbers, letters, and other characters, since some common symbols often appear, in order to distinguish these common symbols from other characters, the preset encoding corresponding to the Chinese character type is set to c, the preset encoding corresponding to the numeric character type is set to d, the preset encoding corresponding to the letter character type is set to a, the preset encoding corresponding to the space character type is set to s, the preset encoding corresponding to the short dash character type is set to -, the preset encoding corresponding to the underscore character type is set to _, and the preset encoding corresponding to other character types is set to o. Further, the statistical quantity of each character type continuously appearing in the target training data is counted, and the target training data is encoded according to the preset encoding and the statistical quantity. For example, a target training data is "Dele_SSD 240G-DLDS1721001090". Each character of "Dele" is encoded as c, "_" is encoded as _, each character of "SSD" is encoded as a, "" is encoded as s, each character of "240" is encoded as d, "G" is encoded as a, "" is encoded as s, "-" is encoded as -, each character of "DLDS" is encoded as a, and each character of "1721001090" is encoded as d. By counting the statistical quantity of each character type continuously appearing, the target training data "Dele_SSD 240G-DLDS1721001090" can be encoded as "2c1_3d1a1s1-4a10d".

[0105] In some embodiments, referring to Figure 3 , each target training data encoding in the encodings of multiple target training data is associated with its corresponding user historical portrait and continuity information (i.e., continuity label 1 or 0), so as to determine training samples, and the training samples are marked with the anomaly pattern labels of known anomaly patterns to form a training set, and the detection model is trained using the training set.

[0106] In some embodiments, any one or several target training data encodings can be selected as target screening conditions to obtain multiple data classes. For example, the encoding corresponding to the same game nickname is used as the target screening condition, and the remaining target training data encodings are associated with the target screening condition to determine the first data class corresponding to the game nickname. It can be understood that by using the remaining target training data encodings as the target screening conditions respectively, all the remaining data classes can be determined.

[0107] Further, determine the intra-class vector corresponding to each target training data code, select any one target training data code and determine it as the target screening condition, and then determine the intra-class distance between each target training data code and the target screening condition in all the remaining target training data codes according to the intra-class vector corresponding to all the remaining target training data codes and the intra-class vector corresponding to the target screening condition, and then obtain all the intra-class distances corresponding to the target screening condition. Further, update the target screening condition to any one target training data code in the remaining target training data codes that have not been selected, repeat the above steps, and determine the intra-class distance between each target training data code in all the remaining target training data codes and the updated target screening condition according to the intra-class vector corresponding to all the remaining target training data codes and the intra-class vector corresponding to the updated target screening condition, until all the intra-class distances corresponding to each target training data code are obtained. Determine the abnormal data corresponding to the abnormal pattern according to the abnormal pattern label, determine the minimum intra-class distance among all the intra-class distances according to the abnormal data, and determine it as the abnormal intra-class distance, and train the detection model using the abnormal intra-class distance so that it can recognize the known abnormal pattern.

[0108] In some embodiments, when determining the intra-class distance of any target training data code, it can be determined as a target screening condition, and the remaining target training data codes are analyzed based on cluster analysis. In the analysis process, the continuity information can be expressed as sum(D(t)) / m, where D(t) represents numerical data marked as 0 or 1, and m is the size of the data class; when performing cluster analysis on IP attribution, the information entropy of the IP attribution can be calculated Where p(x) represents the frequency of a certain IP address. For email account prefixes and player nicknames, the edit distance of the strings of the two data samples before and after can be analyzed, that is, the minimum number of edits required to transform a string into another string by insertion, deletion, or replacement. The weight of each target training data encoding is set according to cluster analysis. When determining the intra-class distance corresponding to each target training data encoding, it can be calculated according to the weights corresponding to different target training data encodings.

[0109] With respect to step S106, after obtaining the trained detection model, the detection model can be used to determine the abnormality of the real-time reservation data, wherein the real-time reservation data can be information such as the player account, player nickname, the server reserved by the player, reservation time, etc. obtained in real time.

[0110] Figure 4 An exemplary flowchart of abnormal data detection combined with online learning according to an embodiment of the present application is shown.

[0111] In some embodiments, reference Figure 4 When a user makes an appointment, each real-time appointment data is timestamped. The real-time appointment data can be written into storage according to the time sequence of the timestamp through distributed processing engines such as Flink and Spark, and marked as unreasoned data. Furthermore, the unreasoned data can be hash mapped, for example, by hash mapping all data or certain fields of an unreasoned data, converting them into a positive integer, and determining the positive integer as the first hash value corresponding to the unreasoned data. Further, according to the hash value corresponding to the unreasoned data, an inference instance is assigned to it, wherein the inference instance is an online abnormal account detection service, and the online detection is composed of multiple identical detection services. A large number of appointment accounts are inferred through multiple inference instances, which can ensure the high availability of the service. Each inference instance corresponds to a second hash value. For example, when the first hash value corresponding to an unreasoned data is 2, the inference instance with the second hash value of 2 is matched in multiple inference instances, and the unreasoned data with the same first hash value and second hash value is assigned to the corresponding inference instance.

[0112] Since the detection model has been trained with abnormal pattern labels corresponding to known abnormal patterns, all reasoning instances can be synchronized when real-time appointment data is detected through the detection model. The synchronized information includes the identified abnormal patterns and the data that have been inferred, ensuring that each reasoning instance will not repeat the reasoning on a certain data that has been inferred in one of the reasoning instances, thereby improving detection efficiency.

[0113] Specifically, for example, there are three reasoning instances that have been assigned unreasoned data, namely A, B and C. Reasoning instance A packages its identified abnormal patterns and the data that have been reasoned and sends them to reasoning instance B and reasoning instance C; reasoning instance B packages its identified abnormal patterns and the data that have been reasoned and sends them to reasoning instance A and reasoning instance C; reasoning instance C packages its identified abnormal patterns and the data that have been reasoned and sends them to reasoning instance A and reasoning instance B, thereby completing the information synchronization between all reasoning instances. According to the timestamp of the unreasoned data, the real-time reservation data of the N time windows before the timestamp is read, and it is determined as the data to be detected. The data to be detected is detected according to the detection model to determine its corresponding abnormality.

[0114] For steps S108 and S110, determine whether the abnormality of the real-time reservation data is greater than the preset first abnormality threshold. If the abnormality is greater than the preset first abnormality threshold, the real-time reservation data is determined to be abnormal data. It should be noted that the abnormal pattern can be a studio reservation pattern obtained during offline training and online reasoning, which is a common manifestation of a batch of reservation accounts. For example, the same MAC address and hard disk information are determined as target screening conditions, and the abnormality determined by the intra-class distance calculated by the remaining features is greater than the preset first abnormality threshold (for example, 10), then it is identified as an abnormal reservation account, and its corresponding real-time reservation data is identified as abnormal data.

[0115] When the service was first launched, the abnormal patterns that had been identified were mainly those learned by the detection model during offline training. As the number of reservations continued to increase, each inference instance continued to identify abnormal accounts. After manual confirmation or after exceeding a certain degree of abnormality, it was determined to be abnormal. This abnormal pattern was set as a new abnormal pattern and added to the abnormal pattern label. Based on online learning technology, the detection model can optimize itself using the abnormal pattern label determined according to the new abnormal pattern, so that the detection model can identify the abnormal data corresponding to the new abnormal pattern during the next recognition.

[0116] Specifically, if it is detected that the abnormality degree corresponding to the real-time appointment data is higher than the preset second abnormality degree threshold, where the first preset abnormality degree threshold is less than the second preset abnormality degree threshold, that is to say, the abnormality degree of the current real-time appointment data has not been trained in the detection model training process and is a new abnormal pattern, then a cluster analysis is performed on the real-time appointment data, and the label corresponding to the new abnormal pattern is determined based on the real-time appointment data corresponding to the abnormality degree exceeding the preset second abnormality degree, and the detection model is updated according to the real-time appointment data and the new abnormal pattern label based on online learning technology.

[0117] For example, during offline training, it is found that under the same MAC address and hard disk information, the abnormality calculated by the other features (such as reservation time continuity, IP location, encoded email account prefix, and encoded player nickname) is greater than the preset first abnormality threshold 10, then the corresponding account is an abnormal reservation account, and the corresponding real-time reservation data is abnormal data, then this mode is an existing abnormal mode. However, with the continuous increase of abnormal accounts, the abnormality of other screening conditions and feature combinations in existing abnormal accounts can be analyzed. For example, under the same IP location service provider, the y abnormality of the encoded email account prefix and the player nickname is greater than the preset abnormality interval. Through manual determination, these abnormal accounts are studio batch reservation accounts, then new abnormal modes are added. Therefore, the difference between offline and online recognition of abnormal modes is that during offline training, due to the limitation of the number of abnormal data, only some abnormal modes can be obtained, and as the number of samples increases, new abnormal modes can be more excavated. Through online learning technology, the detection model can learn new abnormal modes, so that it can identify existing abnormal modes and new abnormal modes.

[0118] In some embodiments, a recovery time node can be set. When it is detected that the detection model stops service or the reasoning instance stops service, the recovery time node and the real-time reservation data within the current moment are reset to unreasoned data according to the recovery time node, and the abnormality degree and detection results of the abnormal data of the real-time reservation data within the recovery time node and the current moment are cleared, that is, the entire detection service is traced back to the recovery time node, and the detection model or reasoning instance that stops service is restarted at the recovery time node. The recovery time node can be any time point. For example, if it is earlier than the service launch time, the service launch time can be set as the recovery time node, and if it is later than the current moment, the current moment can be set as the recovery time node.

[0119] From the above, it can be seen that the abnormal data detection method, device, electronic device and storage medium provided by the present application can obtain historical reservation data, real-time reservation data and abnormal mode labels in the scenario of new game server reservation, and train the detection model through historical reservation data and abnormal mode labels, so that the real-time reservation data can be detected by the detection model before the user creates a role and generates behavioral data, and the abnormal degree of the obtained real-time reservation data is compared with the preset first abnormal degree threshold, and whether the real-time reservation data is abnormal data is further determined. According to the historical reservation data and the real-time reservation data, after feature extraction and conversion, and then through offline training of the historical reservation data and online learning of the real-time reservation data, information of multiple dimensions is aggregated, and the real-time reservation data belonging to the abnormal mode is determined before the user generates any behavioral data, and then the account belonging to the abnormal data is determined, and the present application can identify abnormal data in the new abnormal mode through online learning, improve the accuracy and coverage of abnormal identification, significantly reduce the abnormal identification range of the original reservation data, reduce labor costs, and improve detection efficiency. Further, by setting the recovery time point, the detection problem of multiple reasoning instances processing a large amount of reservation data and service start-stop scenarios is solved, and the reproducibility and consistency of the detection results are guaranteed.

[0120] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the described method.

[0121] It should be noted that the above describes some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0122] Figure 5 A schematic diagram of an exemplary structure of an abnormal data detection device provided in an embodiment of the present application is shown.

[0123] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides an abnormal data detection device.

[0124] refer to Figure 5The abnormal data detection device includes: an acquisition module, a training module, a determination module and a detection module; wherein,

[0125] an acquisition module configured to acquire historical reservation data, real-time reservation data, and abnormal pattern labels;

[0126] A training module, configured to train a detection model based on the historical reservation data and abnormal pattern labels;

[0127] A first determination module is configured to detect the real-time reservation data according to the detection model to determine the abnormality of the real-time reservation data;

[0128] A second determination module is configured to determine whether the abnormality is higher than a preset first abnormality threshold;

[0129] The detection module is configured to determine that the real-time reservation data is abnormal data in response to the abnormality being higher than the preset first abnormality threshold.

[0130] In a possible implementation, the acquisition module is further configured to:

[0131] Get user data;

[0132] Determining account data and historical user behavior data associated with the user data based on the user data;

[0133] Determine the network data and hard disk data bound to the account data according to the account data;

[0134] The historical reservation data is determined based on the user data, account data, historical user behavior data, network data and hard disk data.

[0135] In a possible implementation, the training module is further configured as follows:

[0136] Determine a plurality of target training data and user historical portraits according to the historical reservation data;

[0137] Determine the appointment time according to the historical appointment data;

[0138] Determining continuity information of the plurality of target training data according to the reservation time;

[0139] Encoding the plurality of target training data according to a preset encoding rule to determine a plurality of target training data encodings;

[0140] Associating the plurality of target training data codes, user history profiles and continuity information to determine training samples;

[0141] The detection model is trained according to the training samples and the abnormal pattern labels.

[0142] In a possible implementation, the training module is further configured as follows:

[0143] Sequentially sorting the plurality of target training data according to the appointment time;

[0144] Determine whether the time interval between adjacent target training data is less than a preset time interval according to the reservation time;

[0145] In response to a time interval between the adjacent target training data being less than the preset time interval, determining that the adjacent target training data are continuous;

[0146] In response to the time interval between the adjacent target training data being not less than the preset time interval, it is determined that the adjacent target training data are not continuous.

[0147] In a possible implementation, the training module is further configured as follows:

[0148] Classifying the plurality of target training data to determine at least two character types; wherein each character type corresponds to a different preset code;

[0149] sequentially determining the character type corresponding to each character in the plurality of target training data, and the statistical number of consecutive appearances of the character type;

[0150] Each character in the plurality of target training data is encoded according to the preset encoding and the statistical quantity to determine a plurality of target training data encodings.

[0151] In a possible implementation, the training module is further configured as follows:

[0152] Determine an intra-class vector corresponding to each target training data encoding;

[0153] Select any one of the target training data codes to be determined as a target screening condition, determine the intra-class distance between each of the remaining target training data codes and the target screening condition according to the intra-class vector, and obtain the intra-class distance corresponding to the target screening condition;

[0154] The target screening condition is updated to any one of the target training data codes that have not been selected, and the step of determining the intra-class distance between each of the remaining target training data codes and the target screening condition is repeated until the intra-class distance corresponding to each of the target training data codes is obtained;

[0155] Determining abnormal data according to the abnormal pattern label;

[0156] Determine an abnormal intra-class distance according to the abnormal data and all the intra-class distances;

[0157] The detection model is trained based on the intra-anomaly distance.

[0158] In a possible implementation, the real-time reservation data includes: a timestamp;

[0159] The determination module is further configured as follows:

[0160] storing the real-time reservation data according to the time sequence of the timestamps through a distributed processing engine, and marking the real-time reservation data as unreasoned data;

[0161] Performing hash mapping on the unreasoned data to determine a first hash value corresponding to the unreasoned data;

[0162] Acquire multiple reasoning instances; wherein each of the reasoning instances includes a second Hash value;

[0163] Matching the first hash value and the second hash value to assign the un-inferred data to the inference instance;

[0164] Synchronize all the inference instances, and read the real-time reservation data of N time windows before the timestamp according to the timestamp of the un-inferred data to determine the data to be detected, wherein N is a positive integer;

[0165] The data to be detected is detected according to the detection model to determine the abnormality of the data to be detected.

[0166] In a possible implementation, the device further includes: a restart module;

[0167] The restart module is configured as follows:

[0168] Set the recovery time node;

[0169] In response to the detection model stopping service, resetting the real-time reservation data within the restoration time node and the current time to the un-reasoned data according to the restoration time node;

[0170] Clearing the abnormality degree and abnormal data of the real-time reservation data between the restoration time node and the current time;

[0171] The detection model is restarted according to the recovery time node.

[0172] In a possible implementation, the device further includes: an online learning module;

[0173] The online learning module is further configured to:

[0174] In response to the abnormality being higher than the preset second abnormality threshold, performing cluster analysis on the real-time reservation data;

[0175] Determine a new abnormal pattern label according to the real-time reservation data corresponding to the abnormality degree higher than the second abnormality threshold;

[0176] The detection model is updated according to the real-time reservation data and the new abnormal pattern label based on online learning technology.

[0177] For the convenience of description, the above system is described by dividing the functions into various modules. Of course, when implementing the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0178] The system of the above embodiment is used to implement the corresponding abnormal data detection method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0179] Figure 6 A schematic diagram of an exemplary structure of an electronic device provided in an embodiment of the present application is shown.

[0180] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the abnormal data detection method described in any of the above-mentioned embodiments is implemented. Figure 6 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 610, a memory 620, an input / output interface 630, a communication interface 640, and a bus 650. The processor 610, the memory 620, the input / output interface 630, and the communication interface 640 are connected to each other through the bus 650 in the device.

[0181] The processor 610 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0182] The memory 620 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 620 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 620 and are called and executed by the processor 610.

[0183] The input / output interface 630 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0184] The communication interface 640 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0185] The bus 650 comprises a pathway for transmitting information between the various components of the device (eg, the processor 610, the memory 620, the input / output interface 630, and the communication interface 640).

[0186] It should be noted that, although the above device only shows the processor 610, the memory 620, the input / output interface 630, the communication interface 640 and the bus 650, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0187] The electronic device of the above embodiment is used to implement the corresponding abnormal data detection method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0188] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the abnormal data detection method described in any of the above embodiments.

[0189] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0190] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the abnormal data detection method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0191] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0192] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present application difficult to understand, the known power supply / ground connection with the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform to be implemented in the embodiments of the present application (that is, these details should be fully within the scope of understanding of those skilled in the art). In the case of elaborating specific details (e.g., circuits) to describe exemplary embodiments of the present application, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered to be illustrative rather than restrictive.

[0193] Although the present application has been described in conjunction with specific embodiments of the present application, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0194] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of the present application.

Claims

1. A method for detecting abnormal data, characterized in that: include: Obtain historical reservation data, real-time reservation data and abnormal pattern labels; wherein the real-time reservation data includes the player account, the server reserved by the player and the reservation time; training a detection model based on the historical reservation data and abnormal pattern labels; Detecting the real-time reservation data according to the detection model to determine the abnormality of the real-time reservation data; Determining whether the abnormality is higher than a preset first abnormality threshold; In response to the abnormality being higher than the preset first abnormality threshold, the real-time reservation data is determined to be abnormal data.

2. The method according to claim 1, characterized in that: The obtaining of historical reservation data includes: Get user data; Determining account data and historical user behavior data associated with the user data based on the user data; Determine the network data and hard disk data bound to the account data according to the account data; The historical reservation data is determined based on the user data, account data, historical user behavior data, network data and hard disk data.

3. The method according to claim 1, characterized in that The step of training the detection model according to the historical reservation data and the abnormal pattern label includes: Determine a plurality of target training data and user historical portraits according to the historical reservation data; Determine the appointment time according to the historical appointment data; Determining continuity information of the plurality of target training data according to the reservation time; Encoding the plurality of target training data according to a preset encoding rule to determine a plurality of target training data encodings; Associating the plurality of target training data codes, user history profiles and continuity information to determine training samples; The detection model is trained according to the training samples and the abnormal pattern labels.

4. The method according to claim 3, characterized in that The determining the continuity information of the plurality of target training data according to the reservation time includes: Sequentially sorting the plurality of target training data according to the appointment time; Determine whether the time interval between adjacent target training data is less than a preset time interval according to the reservation time; In response to a time interval between the adjacent target training data being less than the preset time interval, determining that the adjacent target training data are continuous; In response to the time interval between the adjacent target training data being not less than the preset time interval, it is determined that the adjacent target training data are not continuous.

5. The method according to claim 3, characterized in that: The step of encoding the plurality of target training data according to a preset encoding rule to determine a plurality of target training data encodings includes: Classifying the plurality of target training data to determine at least two character types; wherein each character type corresponds to a different preset code; sequentially determining the character type corresponding to each character in the plurality of target training data, and the statistical number of consecutive appearances of the character type; Each character in the plurality of target training data is encoded according to the preset encoding and the statistical quantity to determine a plurality of target training data encodings.

6. The method according to claim 3, characterized in that The step of training the detection model according to the training samples and the abnormal pattern labels includes: Determine an intra-class vector corresponding to each target training data encoding; Select any one of the target training data codes to be determined as a target screening condition, determine the intra-class distance between each of the remaining target training data codes and the target screening condition according to the intra-class vector, and obtain the intra-class distance corresponding to the target screening condition; The target screening condition is updated to any one of the target training data codes that have not been selected, and the step of determining the intra-class distance between each of the remaining target training data codes and the target screening condition is repeated until the intra-class distance corresponding to each of the target training data codes is obtained; Determining abnormal data according to the abnormal pattern label; Determine an abnormal intra-class distance according to the abnormal data and all the intra-class distances; The detection model is trained based on the intra-anomaly distance.

7. The method according to claim 1, characterized in that The real-time reservation data includes: a timestamp; The detecting the real-time reservation data according to the detection model to determine the abnormality of the real-time reservation data includes: storing the real-time reservation data according to the time sequence of the timestamps through a distributed processing engine, and marking the real-time reservation data as unreasoned data; Performing hash mapping on the unreasoned data to determine a first hash value corresponding to the unreasoned data; Acquire multiple reasoning instances; wherein each of the reasoning instances includes a second Hash value; Matching the first hash value and the second hash value to assign the un-inferred data to the inference instance; Synchronize all the inference instances, and read the real-time reservation data of N time windows before the timestamp according to the timestamp of the un-inferred data to determine the data to be detected, wherein N is a positive integer; The data to be detected is detected according to the detection model to determine the abnormality of the data to be detected.

8. The method according to claim 1, characterized in that The method further comprises: Set the recovery time node; In response to the detection model stopping service, resetting the real-time reservation data within the restoration time node and the current time to un-reasoned data according to the restoration time node; Clearing the abnormality degree and abnormal data of the real-time reservation data between the restoration time node and the current time; The detection model is restarted according to the recovery time node.

9. The method according to claim 1, characterized in that: After detecting the real-time reservation data according to the detection model to determine the abnormality of the real-time reservation data, the method further includes: In response to the abnormality being higher than a preset second abnormality threshold, performing cluster analysis on the real-time reservation data; Determine a new abnormal pattern label according to the real-time reservation data corresponding to the abnormality degree higher than the second abnormality threshold; The detection model is updated according to the real-time reservation data and the new abnormal pattern label based on online learning technology.

10. The method according to claim 9, characterized in that The second abnormality threshold is greater than the first abnormality threshold.

11. An abnormal data detection device, characterized in that: include: An acquisition module is configured to acquire historical reservation data, real-time reservation data and abnormal pattern labels; wherein the real-time reservation data includes a player account, a server reserved by the player and a reservation time; A training module, configured to train a detection model based on the historical reservation data and abnormal pattern labels; A first determination module is configured to detect the real-time reservation data according to the detection model to determine the abnormality of the real-time reservation data; A second determination module is configured to determine whether the abnormality is higher than a preset first abnormality threshold; The detection module is configured to determine that the real-time reservation data is abnormal data in response to the abnormality being higher than the preset first abnormality threshold.

12. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 10 is implemented.

13. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to enable the computer to implement the method according to any one of claims 1 to 10.

Images