Detection system and method for managing and controlling abnormal behaviors of users based on micro-batch risks

Abstract

The invention relates to the technical field of marketing risk monitoring, in particular to a system and a method for detecting abnormal behaviors of users based on micro-batch risk management and control, and the system comprises a metadata management module, a real-time monitoring rule management module, a real-time data analysis module and an analysis result management module. According to the method, technical means such as streaming data processing, an event-driven mechanism, a privacy computing technology and anomaly detection based on a mathematical statistical model are introduced, so that the problems of insufficient timeliness, slow response, data lag and the like of an existing regular analysis user account information mode are effectively solved; the advantages of real-time monitoring, quick response, accurate identification, flexible configuration and the like are realized, so that powerful support is provided for an enterprise to construct a more perfect marketing user abnormal behavior monitoring system.

Description

Technical Field

[0001] The present invention relates to the technical field of marketing risk monitoring, and in particular to a detection system and method for abnormal behavior of users based on micro-batch risk management. Background Art

[0002] In the construction of marketing systems, it is crucial to monitor and identify abnormal user behavior. With an efficient monitoring system, by integrating user account information from multiple channels, regular analysis is performed based on preset monitoring rules to accurately identify and mark bad actors such as wool-gathering parties and blacklists, so as to ensure the fairness and purity of marketing activities and effectively resist the financial risks brought by malicious behavior. In addition, the monitoring system can also enhance consumer satisfaction and loyalty, help formulate data-driven marketing strategies, ensure compliance, flexibly respond to market dynamics, promote fair market competition, and strengthen the security of the marketing system.

[0003] In the existing technical solutions, in the marketing decision-making scenario, the method of regularly performing analysis based on preset monitoring rules has the following disadvantages.

[0004] (1) Lack of timeliness: Regularly analyzed user account information cannot provide immediate feedback, resulting in losses when abnormal user behavior is identified.

[0005] (2) Slow response: Regularly analyzed user account information cannot take timely action for rapidly changing fraudulent activities, and lacks the ability to respond immediately.

[0006] (3) Data lag: Analysis based on historical user account information may not accurately reflect current market conditions and user behavior.

[0007] (4) Processing capacity limitations: Traditional monitoring systems are usually unable to process large amounts of real-time data, which limits the depth and breadth of analysis.

[0008] (5) Strategy adjustment delay: There is a time lag between the conversion of analysis results and the adjustment of monitoring rules, which affects the timeliness and effectiveness of monitoring rules.

[0009] (6) Limited predictive capabilities: The lack of ability to predict future trends and behaviors through real-time data limits the foresight to respond to market changes.

[0010] (7) User privacy exposure: Sensitive data is directly exposed in the analysis process of monitoring rules, which is prone to the risk of sensitive data leakage.

[0011] (8) The formulation of monitoring rules lacks effective data support: When formulating monitoring rules, some thresholds are often defined based on experience without effective data support, which affects the accuracy of the analysis results.

[0012] Therefore, a detection system and method for abnormal behavior of users based on micro-batch risk management is proposed. Summary of the invention

[0013] Based on this, it is necessary to provide a detection system and method for abnormal user behavior based on micro-batch risk management to address the above technical problems.

[0014] According to a first aspect of the present invention, a detection system for abnormal user behavior based on micro-batch risk management is provided, the system comprising: a metadata management module, for constructing a dynamic mapping relationship between query conditions and fields of docking channel data according to initialization data script technology or page configuration technology, obtaining real-time user account information, and storing the real-time user account information in a streaming database; and using privacy computing technology to encrypt the real-time user account information stored in a target database to obtain encrypted user account information, and store the encrypted user account information in the target database; a real-time monitoring rule management module, for creating and dynamically adjusting real-time analysis rules, and storing the real-time analysis rules in a cache database; and based on event-driven The real-time data analysis module is used to read the user's real-time account movement information from the streaming database using Flink, and obtain the real-time analysis rules from the cache database to determine whether the user's real-time account movement information exceeds the real-time analysis rules. If so, the user's real-time account movement information is determined to be abnormal behavior data. According to the target business scenario requirements and behavior monitoring type, the user's real-time account movement information is marked, and the marked user's real-time account movement information is stored in the target database corresponding to the target business scenario requirements and behavior monitoring type. If not, the current detection is terminated and the next detection is continued; the analysis result management module is used to visualize the user's encrypted account movement information.

[0015] Optionally, the metadata management module includes: a dynamic mapping relationship construction sub-module, which is used to construct a dynamic mapping relationship between the query conditions and the fields of the docking channel data according to the initialization data script technology or the page configuration technology; a streaming database sub-module, which is used to obtain the user's real-time account information according to the query conditions and the fields of the docking channel data, and store the user's real-time account information in the streaming database; an encryption processing sub-module, which is used to use privacy computing technology to encrypt the user's real-time account information stored in the target database, obtain the user's encrypted account information, and store the user's encrypted account information in the target database.

[0016] Optionally, the dynamic mapping relationship construction submodule includes: an initialization data script submodule, which is used to generate an SQL script containing query conditions according to the data type and English name field, write the SQL script to the streaming database, and execute the SQL script in the streaming database to build a dynamic mapping relationship between the query conditions and the fields of the docking channel data; a page configuration submodule, which is used to select the required query conditions according to the data type and English name field in a visual display interface, and build a dynamic mapping relationship between the query conditions and the fields of the docking channel data.

[0017] Optionally, the encryption processing submodule includes: an encryption rule construction submodule, which is used to construct multiple types of encryption rules according to the encrypted content value required by laws and regulations or encryption requirements, and each encryption rule corresponds to an encrypted content value of one type; a sensitive data checking submodule, which is used to check the initial status of all information fields in the user's real-time dynamic account information according to the encrypted content value required by laws and regulations or encryption requirements, and obtain the checking status of each information field, and the checking status includes whether it has been checked as sensitive data or not checked as sensitive data. For the information field that has been checked as sensitive data, corresponding encryption rules are configured, and the information field that has been checked as sensitive data is desensitized by the configured encryption rules. The desensitization processing submodule is used to adopt the sensitive rules and machine learning model weighing technology to identify and process the information fields that are not checked as sensitive data, obtain the sensitivity score of the information fields that are not checked as sensitive data, compare the sensitivity score with the preset score threshold, and determine whether the sensitivity score exceeds the preset score threshold. If so, the information field that is not checked as sensitive data is determined to be sensitive data, and the corresponding encryption rules are configured. The information field that is not checked as sensitive data is desensitized according to the configured encryption rules to obtain the encrypted dynamic account information of the second user. If not, the current decryption processing operation is terminated and the next decryption processing operation is continued.

[0018] Optionally, the desensitizing processing submodule includes: a sensitive rule identification submodule, which is used to extract multiple types of sensitive rules according to the encrypted content value required by regulations or encryption requirements, and each of the sensitive rules forms a mapping relationship with a type of encrypted content value and a type of encryption rule; a first processing submodule, which is used to calculate the matching degree of all sensitive rules based on the information field that is not checked as sensitive data, using the edit distance algorithm to obtain the matching degree of each sensitive rule, sort the matching degrees of the sensitive rules from high to low, and select the maximum matching degree; a second processing submodule, which is used to vectorize the information field that is not checked as sensitive data to obtain an information vector, input the information vector into a sensitive assessment model, and output the sensitivity score of the corresponding information vector, wherein the sensitive assessment model is based on quantified historical data The historical data set is generated after training the set, and the historical data set is constructed based on the quantified historical information fields and the corresponding sensitive rules; a comparison submodule is used to compare the maximum value of the matching degree with the sensitive score, and determine whether the maximum value of the matching degree exceeds the sensitive score. If so, the maximum value of the matching degree is used as the sensitive score, and if not, the sensitive score is used as the sensitive score; and the sensitive score is compared with a preset score threshold to determine whether the sensitive score exceeds the preset score threshold. If so, the information field that is not checked as sensitive data is determined to be sensitive data, and the corresponding encryption rules are configured. The information field that is not checked as sensitive data is desensitized by the configured encryption rules to obtain the encrypted dynamic account information of the second user. If not, the current decryption processing operation is terminated and the next decryption processing operation is continued.

[0019] Optionally, the real-time monitoring rule management module includes: a first management submodule, used to create real-time analysis rules according to target business scenario requirements and behavior monitoring types, and store the created real-time analysis rules in a cache database; wherein the target business scenario requirements and behavior monitoring types include wool party behavior types and high-risk operation behavior types; a second management submodule, used to analyze and calculate the historical user encrypted dynamic account information stored in the target database based on a mathematical and statistical model, obtain multiple statistical indicator values, and according to the target business scenario requirements and behavior monitoring types, filter out at least one target statistical indicator value from the multiple statistical indicator values ​​to obtain a target statistical indicator combination strategy, dynamically adjust the real-time analysis rules according to the target statistical indicator combination strategy, and store the dynamically adjusted real-time analysis rules in the cache database; an event-driven submodule, used to publish the real-time analysis rules created or dynamically adjusted in the cache database based on an event-driven mechanism.

[0020] Optionally, the real-time data analysis module includes: an acquisition submodule, used to use Flink to read the user's real-time account movement information from the streaming database, and obtain the real-time analysis rules from the cache database; a judgment submodule, used to judge whether the user's real-time account movement information exceeds the real-time analysis rules. If so, the user's real-time account movement information is determined to be abnormal behavior data, and the user's real-time account movement information is marked according to the target business scenario requirements and behavior monitoring type, and the marked user real-time account movement information is stored in the target database corresponding to the target business scenario requirements and behavior monitoring type. If not, the current detection is terminated and the next detection is continued.

[0021] Optionally, the streaming database is one of MQ and Kafka.

[0022] Optionally, the cache database is one of Redis, MySQL, and HBase.

[0023] According to a second aspect of the present invention, a method for detecting abnormal behavior of users based on micro-batch risk management is provided, including: constructing a dynamic mapping relationship between query conditions and fields of docking channel data through a metadata management module according to initialization data script technology or page configuration technology, obtaining real-time user dynamic account information, and storing the real-time user dynamic account information in a streaming database; and using privacy computing technology to encrypt the real-time user dynamic account information stored in a target database to obtain encrypted user dynamic account information, and store the encrypted user dynamic account information in the target database; creating and dynamically adjusting real-time analysis rules through a real-time monitoring rule management module, and storing the real-time analysis rules in a cache database; and based on an event-driven mechanism The real-time analysis rules created in the cache database or dynamically adjusted are published; the real-time data analysis module uses Flink to read the user's real-time account movement information from the streaming database, and obtains the real-time analysis rules from the cache database to determine whether the user's real-time account movement information exceeds the real-time analysis rules. If so, the user's real-time account movement information is determined to be abnormal behavior data. According to the target business scenario requirements and behavior monitoring type, the user's real-time account movement information is marked, and the marked user's real-time account movement information is stored in the target database corresponding to the target business scenario requirements and behavior monitoring type. If not, the current detection is terminated and the next detection is continued; the user's encrypted account movement information is visualized through the analysis result management module.

[0024] The beneficial effects of the present application are as follows: the present application provides a detection system and method for abnormal user behavior based on micro-batch risk management, which has the following beneficial effects: 1. Improved real-time performance: (1) Real-time data processing: Compared with the existing method of periodically analyzing user account information, the present application adopts streaming data processing, which can capture and process data in real time, immediately identify abnormal user behavior, avoid losses caused by analysis delays, and greatly improve timeliness; (2) Real-time effectiveness of rules: The created or dynamically adjusted real-time analysis rules can be synchronized to the real-time data analysis module through the event-driven mechanism. The real-time data analysis module obtains the real-time analysis rules in real time during the execution process, ensuring that the real-time analysis rules take effect immediately without waiting for the next analysis cycle, further improving the response speed; 2. Improved accuracy: (1) Latest data analysis: By processing the latest data in real time, this application can more accurately reflect the current market conditions and user behavior, avoiding data lag problems, thereby improving the accuracy and timeliness of analysis; (2) Dynamic adjustment rules: The target statistical indicator combination strategy can be dynamically adjusted according to the real-time business scenario requirements and behavior monitoring type to adapt to the ever-changing market environment, and can more accurately identify user abnormal behavior, effectively improving the accuracy of identification; (3) Anomaly detection based on mathematical statistics models: Compared with the existing empirically defined monitoring rules, the accuracy of real-time analysis rules is improved; 3. Enhanced flexibility: (1) Flexible configuration: Metadata management The management module supports flexible configuration of the dynamic mapping relationship between the fields of the docking channel data and the query conditions in order to obtain the real-time account information of users, thus improving the flexibility of the system; (2) Custom rules: The real-time monitoring rule management module supports the definition of different real-time analysis rules according to the requirements of different business scenarios and behavior monitoring types, and supports the addition and dynamic adjustment of real-time analysis rules to meet the needs of different business scenarios; 4. Security improvement: (1) Shorten the risk exposure time: Through real-time monitoring and rapid response, this system can effectively shorten the risk exposure time, reduce the probability of risk occurrence, ensure user experience, and improve system security; (2) Data support: The real-time account information of users that exceeds the real-time analysis rules will be recorded in the target database to It can provide data support for marketing and other scenarios, help companies develop more accurate marketing strategies and reduce risks; (3) The introduction of privacy computing technology can realize the encryption of real-time user account information stored in the target database to improve the security and compliance of the system; 5. Efficiency improvement: Adopting an event-driven mechanism, the creation and dynamic adjustment of real-time analysis rules are used as publishing events. In the event monitoring processing logic, the real-time analysis rule update function can be realized, and according to the system configuration, a variety of real-time analysis rule update methods can be realized, such as Redis, MySQL, HBase and other cache database data updates, shortening the effective time of the real-time analysis rules and further improving the overall efficiency of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 is an exemplary system architecture diagram to which the present application can be applied.

[0026] Figure 2 It is a structural diagram of an embodiment of a detection system for abnormal behavior of users based on micro-batch risk management according to the present application.

[0027] Figure 3 It is a flowchart of an embodiment of a method for detecting abnormal behavior of users based on micro-batch risk management according to the present application. DETAILED DESCRIPTION

[0028] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0029] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0030] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0031] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0032] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0033] Terminal devices 101, 102, 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III, Moving Picture Experts Group Audio Layer 3), MP4 (Moving Picture Experts Group Audio Layer IV, Moving Picture Experts Group Audio Layer 4) players, laptop computers and desktop computers, etc.

[0034] The server 105 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal devices 101 , 102 , and 103 .

[0035] It should be noted that the detection system for abnormal user behavior based on micro-batch risk management provided in the embodiment of the present application is generally set in the server / terminal device, and correspondingly, the detection method for abnormal user behavior based on micro-batch risk management is generally executed by the server / terminal device.

[0036] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0037] Continue to refer Figure 2 , shows a schematic diagram of the structure of an embodiment of a detection system 200 for abnormal user behavior based on micro-batch risk management according to the present application. The system effectively solves the problems of insufficient timeliness, slow response, data lag, etc. of the existing regular analysis of user account information by introducing technical means such as streaming data processing, event-driven mechanism, privacy computing technology, and abnormal detection based on mathematical statistics models, and achieves the advantages of real-time monitoring, rapid response, accurate identification, and flexible configuration, thereby providing strong support for enterprises to build a more complete marketing user abnormal behavior monitoring system. The system specifically includes: metadata management module 201, real-time monitoring rule management module 202, real-time data analysis module 203, and analysis result management module 204.

[0038] The metadata management module 201 is used to construct a dynamic mapping relationship between query conditions and fields of docking channel data based on the initialization data script technology or page configuration technology, obtain the user's real-time account information, and store the user's real-time account information in the streaming database; and use privacy computing technology to encrypt the user's real-time account information stored in the target database to obtain the user's encrypted account information, and store the user's encrypted account information in the target database.

[0039] According to one embodiment of the present application, the metadata management module 201 specifically includes: a dynamic mapping relationship construction sub-module, which is used to construct a dynamic mapping relationship between the query conditions and the fields of the docking channel data according to the initialization data script technology or the page configuration technology; a streaming database sub-module, which is used to obtain the user's real-time account information according to the query conditions and the fields of the docking channel data, and store the user's real-time account information in the streaming database; an encryption processing sub-module, which is used to use privacy computing technology to encrypt the user's real-time account information stored in the target database, obtain the user's encrypted account information, and store the user's encrypted account information in the target database.

[0040] According to one embodiment of the present application, the dynamic mapping relationship construction submodule includes: an initialization data script submodule, which is used to generate an SQL script containing query conditions according to the data type and English name field, write the SQL script to the streaming database, and execute the SQL script in the streaming database to build a dynamic mapping relationship between the query conditions and the fields of the docking channel data; a page configuration submodule, which is used to select the required query conditions according to the data type and English name field in a visual display interface, and build a dynamic mapping relationship between the query conditions and the fields of the docking channel data.

[0041] According to an embodiment of the present application, constructing a dynamic mapping relationship between query conditions and fields of docking channel data by initializing a data script submodule may include the following steps.

[0042] 1. Prepare SQL scripts: (1) Common data is the cardholder field information, represented by "C_", such as "C_field", and is generated by batch index interception; (2) Queue data field information is represented by "Q_", such as "Q_field", and is the upstream data provided in the row for event definition; (3) Aggregate data field information (product), distinguished by type, paramType=y.

[0043] 2. Execute SQL script: Execute SQL script in the streaming database.

[0044] 3. Data query: Provides a query interface to query the three types of inserted data based on the English name field.

[0045] According to one embodiment of the present application, a dynamic mapping relationship between query conditions and fields of docking channel data is constructed through a page configuration submodule, including the following steps: in a visual display interface, multiple candidate lists (i.e., query conditions) are set, each candidate list includes a field description of a data type and a corresponding English name field, and the required query conditions can be selected by checking the required query conditions to construct a dynamic mapping relationship between fields of query conditions and docking channel data.

[0046] According to one embodiment of the present application, the streaming database submodule is one of MQ and Kafka.

[0047] According to one embodiment of the present application, the encryption processing submodule includes: an encryption rule construction submodule, which is used to construct multiple types of encryption rules according to the encrypted content value required by laws and regulations or encryption requirements, and each encryption rule corresponds to an encrypted content value of a type; a sensitive data checking submodule is used to check the initial status of all information fields in the user's real-time dynamic account information according to the encrypted content value required by laws and regulations or encryption requirements, and obtain the checking status of each information field, wherein the checking status includes being checked as sensitive data or not being checked as sensitive data, and for the information field that has been checked as sensitive data, a corresponding encryption rule is configured, and the information field that has been checked as sensitive data is checked by the configured encryption rule. Desensitizing processing is performed to obtain the encrypted dynamic account information of the first user; a desensitizing processing submodule is used to adopt sensitive rules and machine learning model weighing technology to identify and process the information fields that are not checked as sensitive data, obtain the sensitivity score of the information fields that are not checked as sensitive data, compare the sensitivity score with the preset score threshold, and determine whether the sensitivity score exceeds the preset score threshold. If so, the information field that is not checked as sensitive data is determined to be sensitive data, and the corresponding encryption rules are configured. The information field that is not checked as sensitive data is desensitized according to the configured encryption rules to obtain the encrypted dynamic account information of the second user. If not, the current decryption processing operation is terminated and the next decryption processing operation is continued.

[0048] According to one embodiment of the present application, the desensitization processing submodule includes: a sensitive rule identification submodule, which is used to extract multiple types of sensitive rules according to the encrypted content value required by regulations or encryption requirements, and each of the sensitive rules forms a mapping relationship with a type of encrypted content value and a type of encryption rule; a first processing submodule, which is used to calculate the matching degree of all sensitive rules based on the information field that is not checked as sensitive data, using the edit distance algorithm to obtain the matching degree of each sensitive rule, sort the matching degrees of the sensitive rules from high to low, and select the maximum matching degree; a second processing submodule, which is used to vectorize the information field that is not checked as sensitive data to obtain an information vector, input the information vector into a sensitive assessment model, and output the sensitivity score of the corresponding information vector, wherein the sensitive assessment model is based on the quantized historical data. The historical data set is generated after training, and the historical data set is constructed based on the quantified historical information fields and the corresponding sensitive rules; a comparison submodule is used to compare the maximum value of the matching degree with the sensitive score, and determine whether the maximum value of the matching degree exceeds the sensitive score. If so, the maximum value of the matching degree is used as the sensitive score, and if not, the sensitive score is used as the sensitive score; and the sensitive score is compared with a preset score threshold to determine whether the sensitive score exceeds the preset score threshold. If so, the information field that is not checked as sensitive data is determined to be sensitive data, and the corresponding encryption rules are configured. The information field that is not checked as sensitive data is desensitized by the configured encryption rules to obtain the encrypted dynamic account information of the second user. If not, the current decryption processing operation is terminated and the next decryption processing operation is continued.

[0049] According to one embodiment of the present application, the real-time account information of users that have completed the detection can be used in multiple scenarios such as subsequent marketing, equity distribution, and user analysis. The present application also provides a method for encrypting the real-time account information of users using privacy computing technology, which includes the following steps.

[0050] First, in the metadata management module 201, a checkbox "Is it sensitive data?" is set to check the initial status of all information fields in the user's real-time account information. According to the laws and regulations, encrypted content values ​​that must not be easily disclosed or have confidentiality requirements can be checked as sensitive data for the corresponding information fields, and corresponding encryption rules can be configured for them. For example, the middle four digits of the mobile phone number field are fuzzy to ensure that these sensitive fields can be accurately identified and protected during the processing. For information fields that are not checked as sensitive data, sensitive data that may exist in their content can be desensitized through sensitive rules and machine learning model weighing technology.

[0051] Secondly, according to the encrypted content value required by regulations or encryption requirements, multiple types of sensitive rules are extracted. For example, the sensitive rule corresponding to the ID card number is a continuous 18-digit number. Common sensitive rules include but are not limited to identity identification (such as ID card number, passport number, driver's license number, etc.), financial data (such as bank account number, transaction amount, payment slip number, etc.), and privacy contact (such as mobile phone number, email address, home address, etc.). The Levenshtein Distance algorithm is used to measure the degree of match between the information field content and each sensitive rule, and the matching degree of each sensitive rule is obtained. The maximum value of all matching degrees is taken as the evaluation of the fit between the information field content and each sensitive rule. It should be noted that the Levenshtein Distance algorithm used in this application is a prior art and will not be repeated here.

[0052] Secondly, the information field is vectorized and converted into a data form that can be efficiently recognized and analyzed by the machine learning model to obtain an information vector, which is then input into the sensitivity assessment model to obtain the sensitivity score output by the model. The sensitivity score comprehensively reflects the characteristic performance and value assessment of the data in the sensitivity assessment model after vectorization. Among them, the sensitivity assessment model is generated after training based on the quantified historical data set, and the historical data set is constructed based on the quantified historical information field and the corresponding sensitivity rules.

[0053] Finally, by weighing the maximum value of the matching degree and the sensitivity score, the maximum value of the two is taken as the sensitivity score for the final judgment. If the sensitivity score is greater than the preset score threshold (such as 65%), the information field is determined to be sensitive data, and the corresponding encryption rules are configured. The information field is desensitized through the configured encryption rules to achieve the goal of "computable but invisible", so that the monitoring result data can provide strong security data support for marketing, customer tags and other scenarios.

[0054] The real-time monitoring rule management module 202 is used to create and dynamically adjust real-time analysis rules, and store the real-time analysis rules in the cache database; and publish the real-time analysis rules created or dynamically adjusted in the cache database based on an event-driven mechanism.

[0055] According to one embodiment of the present application, the real-time monitoring rule management module 202 includes: a first management sub-module, which is used to create real-time analysis rules according to the target business scenario requirements and behavior monitoring types, and store the created real-time analysis rules in the cache database; wherein the target business scenario requirements and behavior monitoring types include wool party behavior types and high-risk operation behavior types; a second management sub-module, which is used to analyze and calculate the historical user encrypted dynamic account information stored in the target database based on a mathematical and statistical model, obtain multiple statistical indicator values, and according to the target business scenario requirements and behavior monitoring types, select at least one target statistical indicator value from the multiple statistical indicator values ​​to obtain a target statistical indicator combination strategy, dynamically adjust the real-time analysis rules according to the target statistical indicator combination strategy, and store the dynamically adjusted real-time analysis rules in the cache database; an event-driven sub-module, which is used to publish the real-time analysis rules created or dynamically adjusted in the cache database based on an event-driven mechanism.

[0056] According to an embodiment of the present application, according to the requirements of the target business scenario, the behavior monitoring types are carefully classified, including but not limited to the wool party behavior type and the high-risk operation behavior type. For each behavior monitoring type, the query conditions required for monitoring will be clearly defined, and the corresponding maintenance and update will be performed in the metadata management module 201. Taking the wool party behavior type as an example, the user's activity browsing page, click count, page stay time, activity participation frequency, and the use of activity rights and interests are usually monitored in real-time. Therefore, the alarm threshold of the reasonable real-time analysis rule will be set for these monitoring data fields. For example, when it is detected that the user's activity participation frequency exceeds 5 times and the activity page stay time is less than 1 second, the system will mark the user's real-time account information, that is, mark the user as a wool party user, and store the user's real-time account information in the wool party database. In addition, taking the high-risk operation behavior type as an example, when it is detected that the time difference of the user's commonly used device change is less than 1 day and the IP address information when the account is logged in changes more than 3 times a day, the system will mark the user's real-time account information, that is, mark the user as a high-risk user, and store the user's real-time account information in the high-risk database. The real-time account information of users stored in the wool party database or high-risk database can provide strong data support for subsequent marketing, equity distribution, user analysis and other scenarios. For example, in marketing activities, wool party users and high-risk users can be excluded from participating to ensure the fairness and effectiveness of the activities.

[0057] According to an embodiment of the present application, by providing a data analysis method based on a mathematical statistics model, it is possible to analyze historical user encrypted account information in a rigorous and scientific manner, establish a normal range model, and obtain multiple statistical indicator values, such as mean, median, standard deviation and other statistical indicators, thereby providing data support for the dynamic adjustment of real-time analysis rules. Taking transaction amount analysis as an example, it is necessary to focus on multi-dimensional user real-time account information, covering transaction amount data such as user consumption amount, transfer amount, recharge amount, and the number of transactions of users in different time spans (such as one day, one week, and one month) (i.e., such transaction frequency data). In actual business scenarios, in the alarm threshold setting of the real-time analysis rules for wool users, the number of activity page views, the number of hot spot clicks, and the page stay time monitoring items can refer to the median combined with a small range of fluctuation values ​​(such as the median ± 15% median) to set the alarm threshold interval of the real-time analysis rules, so as to meet the behavior habits of most users and reduce abnormal misjudgments; for example, in the alarm threshold setting of the real-time analysis rules for high-risk users, a suitable statistical indicator combination strategy can be selected according to the business robustness requirements. If the business focuses on preventing large abnormal transactions, the normal high value is defined by the mean plus several times the standard deviation as the upper limit (such as mean + 3 times the standard deviation). Since extremely large transactions tend to raise the standard deviation, this method can be used to identify rare high-value anomalies. If a universal intermediate state measurement is pursued, the median combined with a small range of fluctuations (such as median ± 10% of the median) is used to set the alarm threshold interval of the real-time analysis rules, so as to meet the regular recharge and consumption habits of most users and reduce abnormal misjudgments.

[0058] According to one embodiment of the present application, the created or dynamically adjusted real-time analysis rules will be synchronously stored in the cache database, which can support multiple options, including but not limited to Redis, MySQL, HBase, etc. Based on the event-driven mechanism (the event-driven mechanism can adopt the event-driven mechanism provided by the SpringBoot framework), the creation, dynamic adjustment and other operations of the real-time analysis rules are used as published events. In the event monitoring processing logic, the real-time analysis rule update function can be implemented to ensure that the newly added / modified real-time analysis rules take effect immediately.

[0059] The real-time data analysis module 203 is used to read the user's real-time account movement information from the streaming database using Flink, and obtain the real-time analysis rules from the cache database, to determine whether the user's real-time account movement information exceeds the real-time analysis rules. If so, the user's real-time account movement information is determined to be abnormal behavior data, and the user's real-time account movement information is marked according to the target business scenario requirements and behavior monitoring type, and the marked user's real-time account movement information is stored in the target database corresponding to the target business scenario requirements and behavior monitoring type. If not, the current detection is terminated and the next detection is continued.

[0060] According to one embodiment of the present application, the real-time data analysis module 203 includes: an acquisition submodule, which is used to use Flink to read the user's real-time account movement information from the streaming database and obtain the real-time analysis rules from the cache database; a judgment submodule, which is used to judge whether the user's real-time account movement information exceeds the real-time analysis rules. If so, the user's real-time account movement information is determined to be abnormal behavior data, and the user's real-time account movement information is marked according to the target business scenario requirements and behavior monitoring type, and the marked user real-time account movement information is stored in the target database corresponding to the target business scenario requirements and behavior monitoring type. If not, the current detection is terminated and the next detection is continued.

[0061] According to one embodiment of the present application, the present application deploys a Flink engineering package developed based on the Java language to a Flink cluster, and uses Kafka as a data source. Each time new user real-time account movement information is received, the real-time analysis rules are obtained from the cache database, and it is determined whether the user's real-time account movement information exceeds the real-time analysis rules. If it exceeds, the user's real-time account movement information is determined to be abnormal behavior data, and according to the behavior monitoring type to which the user's real-time account movement information belongs, the user's real-time account movement information is recorded in a related database.

[0062] The analysis result management module 204 is used to visually display the user's encrypted account information.

[0063] According to an embodiment of the present disclosure, any multiple modules of the metadata management module 201, the real-time monitoring rule management module 202, the real-time data analysis module 203, and the analysis result management module 204 can be combined in one module, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the metadata management module 201, the real-time monitoring rule management module 202, the real-time data analysis module 203, and the analysis result management module 204 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, at least one of the metadata management module 201, the real-time monitoring rule management module 202, the real-time data analysis module 203 and the analysis result management module 204 can be at least partially implemented as a computer program module, which can perform corresponding functions when executed.

[0064] The electronic device according to an embodiment of the present application includes a processor, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded from a storage portion into a random access memory (RAM). The processor may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a dedicated microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor may also include an onboard memory for caching purposes. The processor may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present application.

[0065] In the RAM, various programs and data required for the operation of the electronic device are stored. The processor, ROM, and RAM are connected to each other through a bus. The processor performs various operations of the method flow according to the embodiment of the present application by executing the program in the ROM and / or RAM. It should be noted that the program can also be stored in one or more memories other than ROM and RAM. The processor can also perform various operations of the method flow according to the embodiment of the present application by executing the program stored in the one or more memories.

[0066] According to an embodiment of the present application, the electronic device may further include an input / output (I / O) interface, which is also connected to the bus. The electronic device may also include one or more of the following components connected to the I / O interface: an input part including a keyboard, a mouse, etc.; an output part including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage part including a hard disk, etc.; and a communication part including a network interface card such as a LAN card, a modem, etc. The communication part performs communication processing via a network such as the Internet. The drive is also connected to the I / O interface as needed. Removable media, such as magnetic disks, optical disks, magneto-optical disks, semiconductor memories, etc., are installed on the drive as needed so that the computer program read therefrom is installed into the storage part as needed.

[0067] The present application also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present application is implemented.

[0068] According to an embodiment of the present application, a computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present application, a computer-readable storage medium may include the ROM and / or RAM described above and / or one or more memories other than ROM and RAM.

[0069] The embodiment of the present application also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the item recommendation method provided in the embodiment of the present application.

[0070] When the computer program is executed by the processor, the above functions defined in the system / device of the embodiment of the present application are performed. According to the embodiment of the present application, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0071] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device, a magnetic storage device, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of a signal on a network medium, and downloaded and installed through a communication part, and / or installed from a removable medium. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0072] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part, and / or installed from a removable medium. When the computer program is executed by the processor, the above-mentioned functions defined in the system of the embodiment of the present application are executed. According to the embodiment of the present application, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0073] According to an embodiment of the present application, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0074] Further references Figure 3 , as a response to the above Figure 2 The present application provides an implementation method of a method for detecting abnormal behavior of users based on micro-batch risk management. Figure 2 The system embodiment shown corresponds to the embodiment shown.

[0075] like Figure 3 As shown, the present embodiment describes a method for detecting abnormal behavior of users based on micro-batch risk management, and the method includes the following steps.

[0076] S1. Through the metadata management module, according to the initialization data script technology or page configuration technology, a dynamic mapping relationship between the query conditions and the fields of the docking channel data is constructed to obtain the user's real-time account information, and store the user's real-time account information in the streaming database; and using privacy computing technology, the user's real-time account information stored in the target database is encrypted to obtain the user's encrypted account information, and the user's encrypted account information is stored in the target database.

[0077] S2. Create and dynamically adjust real-time analysis rules through the real-time monitoring rule management module, and store the real-time analysis rules in the cache database; and publish the real-time analysis rules created or dynamically adjusted in the cache database based on the event-driven mechanism.

[0078] S3. Use Flink through the real-time data analysis module to read the user's real-time account movement information from the streaming database, and obtain the real-time analysis rules from the cache database to determine whether the user's real-time account movement information exceeds the real-time analysis rules. If so, the user's real-time account movement information is determined to be abnormal behavior data. According to the target business scenario requirements and behavior monitoring type, the user's real-time account movement information is marked, and the marked user's real-time account movement information is stored in the target database corresponding to the target business scenario requirements and behavior monitoring type. If not, end the current detection and continue to the next detection.

[0079] S4. Visually display the user's encrypted account information through the analysis result management module.

[0080] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0081] The above contents are further detailed descriptions of the present invention in combination with specific implementation methods, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the scope of protection of the present invention.

Claims

1. A detection system for abnormal user behavior based on micro-batch risk management, characterized in that: The system includes: The metadata management module is used to construct a dynamic mapping relationship between query conditions and fields of docking channel data based on the initialization data script technology or page configuration technology, obtain the user's real-time dynamic account information, and store the user's real-time dynamic account information in the streaming database; and use privacy computing technology to encrypt the user's real-time dynamic account information stored in the target database to obtain the user's encrypted dynamic account information, and store the user's encrypted dynamic account information in the target database; A real-time monitoring rule management module is used to create and dynamically adjust real-time analysis rules and store the real-time analysis rules in a cache database; and based on an event-driven mechanism, publish the real-time analysis rules created or dynamically adjusted in the cache database; The real-time data analysis module is used to read the user's real-time account information from the streaming database using Flink, and obtain the real-time analysis rules from the cache database to determine whether the user's real-time account information exceeds the real-time analysis rules. If so, the user's real-time account information is determined to be abnormal behavior data, and the user's real-time account information is marked according to the target business scenario requirements and behavior monitoring type, and the marked user's real-time account information is stored in the target database corresponding to the target business scenario requirements and behavior monitoring type. If not, the current detection is terminated and the next detection is continued; The analysis result management module is used to visually display user encrypted account information.

2. According to claim 1, a detection system for abnormal behavior of users based on micro-batch risk management and control is characterized in that: The metadata management module includes: The dynamic mapping relationship building submodule is used to build a dynamic mapping relationship between query conditions and fields of docking channel data based on the initialization data script technology or page configuration technology; The streaming database submodule is used to obtain the user's real-time account information based on the query conditions and the dynamic mapping relationship between the fields of the docking channel data, and store the user's real-time account information in the streaming database; The encryption processing submodule is used to adopt privacy computing technology to encrypt the user's real-time dynamic account information stored in the target database, obtain the user's encrypted dynamic account information, and store the user's encrypted dynamic account information in the target database.

3. According to claim 2, a detection system for abnormal behavior of users based on micro-batch risk management and control is characterized in that: The dynamic mapping relationship construction submodule includes: Initialize the data script submodule, which is used to generate an SQL script containing query conditions according to the data type and English name field, write the SQL script to the streaming database, and execute the SQL script in the streaming database to build a dynamic mapping relationship between the query conditions and the fields of the docking channel data; The page configuration submodule is used to select the required query conditions according to the data type and English name field in the visual display interface, and to build a dynamic mapping relationship between the query conditions and the fields of the docking channel data.

4. According to claim 2, a detection system for abnormal behavior of users based on micro-batch risk management and control is characterized in that: The encryption processing submodule includes: The encryption rule construction submodule is used to construct multiple types of encryption rules according to the encrypted content value required by laws and regulations or encryption requirements, and each encryption rule corresponds to one type of encrypted content value; The sensitive data checking submodule is used to check the initial status of all information fields in the user's real-time account information according to the encrypted content value required by the law or encryption requirements, and obtain the checking status of each information field, wherein the checking status includes being checked as sensitive data or not being checked as sensitive data, and for the information field checked as sensitive data, configure the corresponding encryption rule, and perform desensitization processing on the information field checked as sensitive data according to the configured encryption rule, to obtain the first user's encrypted account information; The desensitization processing submodule is used to adopt sensitive rules and machine learning model weighing technology to identify and process information fields that are not checked as sensitive data, obtain the sensitivity score of the information field that is not checked as sensitive data, compare the sensitivity score with a preset score threshold, and determine whether the sensitivity score exceeds the preset score threshold. If so, the information field that is not checked as sensitive data is determined to be sensitive data, and the corresponding encryption rules are configured. The information field that is not checked as sensitive data is desensitized according to the configured encryption rules to obtain the encrypted account information of the second user. If not, the current decryption processing operation is terminated and the next decryption processing operation is continued.

5. According to claim 4, a detection system for abnormal behavior of users based on micro-batch risk management and control is characterized in that: The desensitization processing submodule includes: A sensitive rule identification submodule is used to extract multiple types of sensitive rules according to the encrypted content value required by regulations or encryption requirements, and each of the sensitive rules is mapped to a type of encrypted content value and a type of encryption rule; The first processing submodule is used to calculate the matching degree of all sensitive rules based on the information fields that are not selected as sensitive data by using the edit distance algorithm to obtain the matching degree of each sensitive rule, sort the matching degrees of the sensitive rules from high to low, and select the maximum matching degree; A second processing submodule is used to vectorize the information fields that are not selected as sensitive data to obtain information vectors, input the information vectors into a sensitivity assessment model, and output a sensitivity score corresponding to the information vector, wherein the sensitivity assessment model is generated after training based on the quantized historical data set, and the historical data set is constructed based on the quantized historical information fields and corresponding sensitivity rules; The comparison submodule is used to compare the maximum value of the matching degree with the sensitivity score to determine whether the maximum value of the matching degree exceeds the sensitivity score. If so, the maximum value of the matching degree is used as the sensitivity score; if not, the sensitivity score is used as the sensitivity score; and the sensitivity score is compared with a preset score threshold to determine whether the sensitivity score exceeds the preset score threshold. If so, the information field that is not checked as sensitive data is determined to be sensitive data, and the corresponding encryption rules are configured. The information field that is not checked as sensitive data is desensitized according to the configured encryption rules to obtain the encrypted dynamic account information of the second user. If not, the current decryption processing operation is terminated and the next decryption processing operation is continued.

6. According to claim 1, a detection system for abnormal user behavior based on micro-batch risk management and control is characterized in that: The real-time monitoring rule management module includes: The first management submodule is used to create real-time analysis rules according to the target business scenario requirements and behavior monitoring types, and store the created real-time analysis rules in a cache database; wherein the target business scenario requirements and behavior monitoring types include wool party behavior types and high-risk operation behavior types; The second management submodule is used to analyze and calculate the historical user encrypted dynamic account information stored in the target database based on the mathematical statistics model, obtain multiple statistical indicator values, select at least one target statistical indicator value from the multiple statistical indicator values ​​according to the target business scenario requirements and behavior monitoring type, obtain the target statistical indicator combination strategy, dynamically adjust the real-time analysis rules according to the target statistical indicator combination strategy, and store the dynamically adjusted real-time analysis rules in the cache database; The event-driven submodule is used to publish the real-time analysis rules created or dynamically adjusted in the cache database based on the event-driven mechanism.

7. According to claim 1, a detection system for abnormal user behavior based on micro-batch risk management and control is characterized in that: The real-time data analysis module includes: The acquisition submodule is used to read the user's real-time account information from the streaming database using Flink, and obtain the real-time analysis rules from the cache database; The judgment submodule is used to judge whether the user's real-time account movement information exceeds the real-time analysis rules. If so, the user's real-time account movement information is determined to be abnormal behavior data. According to the target business scenario requirements and behavior monitoring type, the user's real-time account movement information is marked, and the marked user's real-time account movement information is stored in the target database corresponding to the target business scenario requirements and behavior monitoring type. If not, the current detection is terminated and the next detection is continued.

8. According to claim 1, a detection system for abnormal user behavior based on micro-batch risk management and control is characterized in that: The streaming database is one of MQ and Kafka.

9. The system for detecting abnormal behavior of users based on micro-batch risk management according to claim 1 is characterized in that: The cache database is one of Redis, MySQL, and HBase.

10. A method for detecting abnormal behavior of users based on micro-batch risk management, characterized in that: include: By using the metadata management module, a dynamic mapping relationship between query conditions and fields of docking channel data is constructed according to the initialization data script technology or page configuration technology to obtain the user's real-time dynamic account information, and store the user's real-time dynamic account information in the streaming database; and by using the privacy computing technology, the user's real-time dynamic account information stored in the target database is encrypted to obtain the user's encrypted dynamic account information, and the user's encrypted dynamic account information is stored in the target database; Create and dynamically adjust real-time analysis rules through the real-time monitoring rule management module, and store the real-time analysis rules in the cache database; And based on the event-driven mechanism, publish the real-time analysis rules created or dynamically adjusted in the cache database; The real-time data analysis module uses Flink to read the real-time account information of users from the streaming database, and obtains the real-time analysis rules from the cache database to determine whether the real-time account information of users exceeds the real-time analysis rules. If so, the real-time account information of users is determined to be abnormal behavior data. According to the target business scenario requirements and behavior monitoring type, the real-time account information of users is marked, and the marked real-time account information of users is stored in the target database corresponding to the target business scenario requirements and behavior monitoring type. If not, the current detection is terminated and the next detection is continued; The user's encrypted account information is visualized through the analysis result management module.

Images