Abnormal number identification method and device, electronic equipment, storage medium and product

Abstract

The invention provides an abnormal number identification method and device, electronic equipment, a storage medium and a product, and the method comprises the steps: extracting call features of a user number in a current time slice from user call ticket data; inputting the call characteristics of the user number in the current time slice into an abnormal number identification model based on reinforcement learning to obtain an abnormal number identification result; wherein the reward function of the abnormal number recognition model based on reinforcement learning comprises a time reward module which is used for giving model punishment feedback under the condition that the model judges whether the user number is an abnormal number or not based on the call characteristics of the user number in the future time slice. According to the method, the time reward module is designed in the reward function to restrain the timeliness of the model for identifying the abnormal number, so that the probability of delayed identification of the model is reduced, the abnormal number can be identified timely and quickly, and user loss is avoided.

Description

Technical Field

[0001] The present invention relates to the field of information security technology, and in particular to an abnormal number identification method, device, electronic device, storage medium and product. Background Art

[0002] The development of communications technology has brought great convenience to people's work and lives. However, this has also led to an increasing prevalence of various abnormal behaviors, resulting in significant financial losses. Currently, operators have designed and deployed various abnormal behavior detection and recognition algorithms to identify these behaviors and prevent them from continuing by disabling the communication function of calling cards.

[0003] However, existing abnormal behavior detection and identification models fail to consider time factors during modeling, which can lead to late identification and shutdown of anomalous numbers, after they have already engaged in anomalous behavior. Specifically, existing abnormal behavior detection and identification models primarily focus on comprehensive coverage of anomalous numbers, without considering the timing of model identification. As a result, while existing models offer high coverage of anomalous numbers, most of the identified numbers are identified after the anomalous behavior occurs (a few anomalous behaviors occur after the number has been identified and shut down). Ultimately, both late identification and no identification can lead to financial losses for users. Therefore, identifying anomalous numbers promptly and quickly is a pressing issue. Summary of the Invention

[0004] The present invention provides a method, device, electronic device, storage medium and product for identifying abnormal numbers, which are used to solve the defects in the prior art that the model identifies abnormal numbers too late or fails to identify them, resulting in user property losses, and realize timely and rapid identification of abnormal numbers.

[0005] The present invention provides a method for identifying abnormal numbers, comprising: Extract the call characteristics of the user number in the current time slice from the user call record data; Input the call features of the user number in the current time slice into the abnormal number recognition model based on reinforcement learning to obtain the abnormal number recognition result; Among them, the reward function of the abnormal number recognition model based on reinforcement learning includes a time reward module, which is used to give the model penalty feedback when the model determines whether the user number is an abnormal number based on the call characteristics of the user number in the future time slice.

[0006] According to an abnormal number identification method provided by the present invention, the reward function is used to: If the model determines that the number of a normal user is a normal number and does not perform any action, or determines that the number of an abnormal user is an abnormal number and performs any action, a positive feedback of +1 is given to the model; If the model determines that the number of a normal user is an abnormal number and performs a treatment action, or determines that the number of an abnormal user is a normal number and does not perform a treatment action, a negative feedback of -1 is given to the model; In the case where the model needs to determine whether a user number is an abnormal number based on the call characteristics of the user number in the future time slice, and the decision is made late, the model is given penalty feedback.

[0007] According to an abnormal number identification method provided by the present invention, providing penalty feedback to the model includes: Based on the number of consecutive determinations within a preset time period that the model determines based on future call characteristics whether the user number is an abnormal number, a preset function is used to calculate a first function value; the function value of the preset function increases as the number of consecutive determinations increases; Calculating a second function value using the preset function based on the number of time slices within the preset time period; Determining a delay penalty number that increases with the number of delays based on the first function value and the second function value; Penalty feedback is determined based on an adjustment factor for balancing timeliness and accuracy and the delay penalty number.

[0008] According to an abnormal number recognition method provided by the present invention, the abnormal number recognition model based on reinforcement learning includes an estimation model and a target model; after obtaining the abnormal number recognition result, the method further includes: Executing an action corresponding to the abnormal number identification result on the user number; After receiving feedback from the external environment of the model interaction, the parameters of the estimation model are updated based on the target value output by the target model; the target value is obtained by inputting the call characteristics of the user number in the next time slice into the target model; After the estimation model is optimized and updated for a preset number of times, the parameters of the estimation model are assigned to the parameters of the target model to update the parameters of the target model.

[0009] According to an abnormal number recognition method provided by the present invention, the action-value function of the abnormal number recognition model based on reinforcement learning is implemented using an online random forest model, and the sub-decision tree of the online random forest model is updated and iterated in the following manner: Based on a preset call feature division method, the original node in the sub-decision tree is divided into at least two sub-nodes; Based on the true values and predicted values corresponding to the call feature samples falling into the original node and falling into each sub-node, respectively calculating the mean square error of the original node and each sub-node; Calculating the information gain after node division based on the mean square error between the original node and each of the child nodes; When the number of call feature samples falling into the original node is greater than the minimum number of split samples, and the information gain after node division is greater than the minimum split information gain, the original node is split, and based on the preset call feature division method, the call feature samples falling into the original node are allocated to child nodes.

[0010] According to an abnormal number identification method provided by the present invention, the number of times each sub-decision tree receives each arriving call feature sample is a target number, and the target number is a random number generated by Poisson distribution.

[0011] According to an abnormal number recognition method provided by the present invention, the sub-decision tree of the online random forest model is optimized in the following manner: Calculate the out-of-bag error of each sub-decision tree; For each sub-decision tree, if the out-of-bag error of the sub-decision tree is greater than the error upper limit and the age of the sub-decision tree is greater than the discard upper limit, the sub-decision tree is discarded and a new sub-decision tree is initialized.

[0012] The present invention also provides an abnormal number identification device, comprising: The feature extraction module is used to extract the call features of the user number in the current time slice from the user call record data; An abnormality identification module, used to input the call characteristics of the user number in the current time slice into the abnormal number identification model based on reinforcement learning to obtain an abnormal number identification result; Among them, the reward function of the abnormal number recognition model based on reinforcement learning includes a time reward module, which is used to give the model penalty feedback when the model determines whether the user number is an abnormal number based on the call characteristics of the user number in the future time slice.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, any of the above-described abnormal number identification methods is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described abnormal number identification methods.

[0015] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned abnormal number identification methods.

[0016] The abnormal number identification method, device, electronic device, storage medium and product provided by the present invention extract the call characteristics of the user number in the current time slice from the user call record data, and input the call characteristics of the user number in the current time slice into the abnormal number identification model based on reinforcement learning to obtain the abnormal number identification result. Therefore, in the abnormal number identification process, by designing the abnormal number identification model based on reinforcement learning and designing a time reward module in the reward function, when the model determines that it is necessary to judge whether the user number is an abnormal number based on the call characteristics of the user number in the future time slice, a certain penalty feedback can be given to the model. Therefore, by constraining the timeliness of the model in identifying abnormal numbers, the probability of delayed recognition of the model is reduced, and abnormal numbers can be identified in a timely and rapid manner, which is conducive to timely avoiding user losses. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 The figure is a flow chart of the abnormal number identification method provided by an embodiment of the present invention.

[0019] Figure 2 It is a structural diagram of an abnormal number identification model provided by an embodiment of the present invention.

[0020] Figure 3 It is a structural diagram of an abnormal number identification device provided by an embodiment of the present invention.

[0021] Figure 4 It is a structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0022] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0023] In the description of the embodiments of the present invention, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0024] Figure 1 Schematic diagram of the process of identifying abnormal numbers provided by an embodiment of the present invention. Figure 1 The embodiment of the present invention provides a method for identifying abnormal numbers, which may include the following steps: Step 101: Extract the call characteristics of the user number in the current time slice from the user call record data.

[0025] It should be noted that the execution entity of the abnormal number identification method provided in the embodiments of the present invention can be an electronic device, a component of an electronic device, an integrated circuit, or a chip. The electronic device can be a mobile electronic device or a non-mobile electronic device. For example, a mobile electronic device can be a mobile phone, tablet computer, laptop computer, PDA, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA). The non-mobile electronic device can be a server, network attached storage (NAS), personal computer (PC), television, ATM, or self-service machine, etc., which is not specifically limited in the embodiments of the present invention. The following description of the embodiments of the present invention uses a server as the execution entity.

[0026] It should be noted that abnormal users may use abnormal numbers to call other users, thereby engaging in abnormal behavior towards other users during the call. Embodiments of the present invention can identify the numbers of abnormal users (i.e., abnormal numbers) that pose a risk of abnormal behavior by extracting the call characteristics of each user from all user call records.

[0027] In this embodiment of the present invention, all user call record data received within a specific time interval can be considered as user call record data for a time slice. Specifically, data aggregation analysis can be performed on the user call record data for the current time slice, and characteristic statistical values can be extracted from the call record data to obtain the call characteristics of the user number in the current time slice t, thereby identifying whether the user number is an abnormal number.

[0028] In some embodiments, the call characteristics may include at least one of call frequency, called frequency, dispersion, average call interval duration, average call duration, connection rate, average off-hook ringing duration, and called release rate.

[0029] Step 102: Input the call characteristics of the user number in the current time slice into the abnormal number recognition model based on reinforcement learning to obtain the abnormal number recognition result; wherein, the reward function of the abnormal number recognition model based on reinforcement learning includes a time reward module, which is used to give the model penalty feedback when the model determines that it is necessary to determine whether the user number is an abnormal number based on the call characteristics of the user number in the future time slice.

[0030] In the general reinforcement learning framework, for any state s in the state space S, the agent can Select appropriate actions from the action space A To execute and receive the feedback reward from the environment for this action , and according to the state transfer function Update Status The optimization goal of this framework is to find the optimal strategy The cumulative feedback rewards received Maximize, that is, maximize ,in Represents the time discount factor, which is used to control the impact of rewards at different time points on the final reward and. This framework can be optimized based on the action value function. The action value function (also known as the Q value function) is defined as the expected cumulative feedback reward that can be achieved under the premise of a given action and state, that is, According to the Bellman equation, the Q-value function can be decomposed into the current reward and the subsequent Q-value, that is, If the optimal Q-value function is known, the optimal strategy is to greedily execute the action that maximizes the Q-value, that is: ; In the abnormal behavior identification scenario, the abnormal number identification model can determine whether there is an abnormal behavior risk based on the call characteristics of the user number, and make a disposal action or no disposal action. The implementation of the disposal action may be due to the user's complaint and call restoration caused by the abnormal number identification model's incorrect identification, or it may be because the outbound call follow-up after the disposal confirms that the user has committed abnormal behavior; not taking the disposal action may lead to the emergence of victimized users. In order to allow the abnormal number identification model to be optimized in real time based on the feedback results of the external environment and to achieve a balance between the timeliness and accuracy of identification, the embodiment of the present invention designs an abnormal number identification model based on the reinforcement learning framework.

[0031] In the embodiment of the present invention, the user number state space S can be composed of the call feature values of each number in each time slice; the action space A can be composed of three actions, namely, the non-processing action after determining that the user number is a normal number , Actions to take after determining that the user number is an abnormal number , and non-handling actions that require further analysis of subsequent time series characteristics (i.e., call characteristics of the user number in the future time slice) to determine .

[0032] In the abnormal behavior scenario, once the user with abnormal behavior risk (i.e., abnormal user) is identified and dealt with, their subsequent user status no longer needs to be concerned, so their state transition function can be: ; To address the defect that the current abnormal behavior recognition model pays little attention to timeliness, resulting in a high coverage rate of abnormal numbers but most of them are recognized too late, the embodiment of the present invention designs a time reward module in the reward function to constrain the timeliness of the model's recognition of abnormal numbers. When the abnormal number recognition model determines whether a user number is an abnormal number based on the call characteristics of the user number in the future time slice, a certain penalty feedback is given to the model, thereby reducing the probability of delayed recognition by the model, identifying abnormal numbers in a timely and rapid manner, and preventing other users from suffering from abnormal behavior.

[0033] The embodiment of the present invention extracts the call features of the user number in the current time slice from the user call record data, and inputs the call features of the user number in the current time slice into the abnormal number recognition model based on reinforcement learning to obtain the abnormal number recognition result. Therefore, in the abnormal number recognition process, by designing the abnormal number recognition model based on reinforcement learning and designing a time reward module in the reward function, when the model determines that it is necessary to judge whether the user number is an abnormal number based on the call features of the user number in the future time slice, a certain penalty feedback can be given to the model. Therefore, by constraining the timeliness of the model in identifying abnormal numbers, the probability of delayed recognition of the model is reduced, and the abnormal number can be identified in a timely and rapid manner, which is conducive to timely avoiding user losses.

[0034] In an optional embodiment, the reward function can be used to: give the model positive feedback of +1 when the model determines that the number of a normal user is a normal number and no disposal action is performed, or determines that the number of an abnormal user is an abnormal number and a disposal action is performed; give the model negative feedback of -1 when the model determines that the number of a normal user is an abnormal number and a disposal action is performed, or determines that the number of an abnormal user is a normal number and no disposal action is performed; give the model penalty feedback when the model determines that it is necessary to determine whether the user number is an abnormal number based on the call characteristics of the user number in a future time slice, and delays making a decision.

[0035] In an embodiment of the present invention, when the recognition result of the abnormal number recognition model based on reinforcement learning is accurate, that is, the model does not take any action after determining that the number of a normal user is a normal number, or takes any action after determining that the number of an abnormal user is an abnormal number (in implementation, numbers that have not been restored for a period of time can be regarded as abnormal numbers, and numbers that have no risk of abnormal behavior for a period of time can be regarded as normal numbers), a positive feedback of +1 can be given to the model.

[0036] Among them, normal users refer to users who do not have the risk of abnormal behavior, and abnormal users refer to users who have the risk of abnormal behavior.

[0037] In an embodiment of the present invention, when the recognition result of the abnormal number recognition model based on reinforcement learning is wrong, that is, the model takes action after determining the number of a normal user as an abnormal number, or does not take action after determining the number of an abnormal user as a normal number (which corresponds to the scenario where the user makes a reasonable complaint and the relevant department receives a report from the user who has suffered abnormal behavior during implementation), the model is given a negative feedback of -1.

[0038] In an embodiment of the present invention, if the model needs to observe more subsequent data (i.e., it needs to be based on the call characteristics of the user number in the future time slice) to determine whether the user number is an abnormal number with abnormal behavior, and decides to delay making a decision (i.e., no handling action is taken at present), then the model will be given penalty feedback.

[0039] The embodiment of the present invention provides the model with a concept of time by designing a time reward module in the reward function. When the abnormal number recognition model determines whether a user number is an abnormal number based on the call characteristics of the user number in a future time slice (i.e., subsequent time series characteristics), and no disposal action is currently performed, a certain penalty feedback is given to the model, thereby reducing the probability of delayed recognition of the model, and being able to identify abnormal numbers in a timely and rapid manner, thereby preventing other users from suffering from the abnormal behavior of abnormal users in a timely manner.

[0040] In an optional embodiment, providing the model with penalty feedback may specifically include: Step S11, based on the number of consecutive determinations within a preset time period that the model determines based on future call characteristics whether the user number is an abnormal number, a preset function is used to calculate a first function value; the function value of the preset function increases as the number of consecutive determinations increases; Step S12, calculating a second function value using the preset function based on the number of time slices within the preset time period; Step S13, determining a delay penalty number that increases with the number of delays based on the first function value and the second function value; Step S14: Determine penalty feedback based on the adjustment factor for balancing timeliness and accuracy and the delay penalty number.

[0041] In the design of the reward function, the abnormal behavior identification business requires not only the fastest possible identification of abnormal numbers, but also the timeliness of the abnormal number identification model to prevent further users from being exposed to abnormal behavior caused by abnormal numbers calling other users. Improving the model's timeliness also requires improving its accuracy to avoid the negative impact of misidentification on users. However, improving the timeliness results in the model observing less time series data, which reduces the model's accuracy.

[0042] The reward function of the embodiment of the present invention can not only improve the timeliness and accuracy of the model at the same time, but also achieve a balance between the timeliness and accuracy. The reward function can be specifically as follows: ; Among them, the penalty feedback given to the model is a negative feedback that increases with the number of delays. ; The default function is , is a coefficient greater than 1, the first function value is l Substitute the preset function to get , the second function value is obtained by substituting L into the preset function .

[0043] This reward can include and Two parts: Part 1 It is an adjustment factor used to balance timeliness and accuracy. When timeliness is more important than accuracy, it can be increased. value, increase the penalty for failure to meet timeliness; Part II is the delay penalty number, which is used to achieve the effect of increasing the penalty when the delay increases. For continuous execution The number of times, and The number of time slices within the preset time period is used to regularize the delay penalty number.

[0044] The preset time period can be set according to actual needs, for example, it can be one day.

[0045] In the embodiment of the present invention, when a certain penalty is given to the model, the penalty increases with time, thereby limiting the judgment time of the model and improving the recognition timeliness of the model.

[0046] The embodiment of the present invention designs an adjustment factor for balancing the timeliness and accuracy of the model in the time reward module of the reward function to adjust the penalty intensity of the model's incorrect recognition and late recognition, thereby achieving a dynamic self-balancing of "early recognition" and "accurate recognition".

[0047] In an optional embodiment, the abnormal number recognition model based on reinforcement learning includes an estimation model and a target model; after obtaining the abnormal number recognition result, the method may further include: Step S21, performing an action corresponding to the abnormal number identification result on the user number; Step S22, after receiving the action execution feedback result from the external environment of the model interaction, updating the parameters of the estimation model based on the target value output by the target model; the target value is obtained by inputting the call characteristics of the user number in the next time slice into the target model; Step S23 , after the estimation model is optimized and updated for a preset number of times, the parameters of the estimation model are assigned to the parameters of the target model to update the parameters of the target model.

[0048] The existing framework requires continuous human intervention to optimize the model, rather than automatic optimization in real time. This results in the abnormal number identification model being unable to keep pace with the complex and ever-changing developments in abnormal behavior. Specifically, due to the volatile and adversarial nature of abnormal telecommunications behavior, the abnormal identification model needs to be promptly optimized and iterated based on unrecognized or misidentified cases. Under the current technical framework, model iteration requires technicians to periodically train and iterate on unrecognized or misidentified cases, or when the model's performance deteriorates. However, this approach is not only labor-intensive but also struggles to keep pace with the rapid updates and iterations required by the diverse and diverse methods used by abnormal users to implement abnormal behavior at an industrial scale.

[0049] To address these issues, the present invention constructs a model interaction environment and an online random forest model, automatically optimizing the abnormal number recognition model through an "interaction-feedback" model. Specifically, by integrating users, outbound call callbacks, and relevant departments into the model's external interaction environment, the model can learn new abnormal number characteristics through self-service interaction with the external environment, thus achieving real-time adaptive optimization of the abnormal number recognition model.

[0050] Figure 2 This is a schematic diagram of the structure of the abnormal number recognition model provided by the embodiment of the present invention. Figure 2 In an embodiment of the present invention, the abnormal number recognition model based on reinforcement learning may include an estimation model and a target model. The estimation model may output the abnormal number recognition result (i.e., the estimated value) for the current time slice and perform a corresponding action. The action result may then be sent to the external environment, thereby receiving feedback from the external environment regarding the action execution. The parameters of the estimation model may then be updated based on the feedback from the external environment and the target value output by the target model.

[0051] Specifically, in the model reinforcement learning framework, the modeling of the Q-value function can be defined as , indicating that given the Q value function parameters Under the premise that the user number state is s, the Q value obtained by taking action a. According to the reinforcement learning theory, the output value of the abnormal number recognition model can be the estimated value , and the target value It can be: ; in, It can be used to predict the status of the user number in the next time slice, so the model needs to continuously learn and optimize parameters , making and The gap is narrowed. Since both the estimated value and the target value have parameters , which makes it difficult to converge during model training, so an estimation model can be designed With target model Two models with the same structure. During the update and iteration process, only the estimated model is optimized based on real-time data. Parameters , target model Parameters Remain unchanged.

[0052] In the embodiment of the present invention, after the estimation model is optimized and updated for a preset number of times, the estimation model can be Parameters Assign to the target model Parameters , thereby updating the parameters of the target model and entering the next batch of updates. This method cuts off the correlation between the estimated Q value and the target Q value, thereby improving the convergence of the learning process.

[0053] In an optional embodiment, the action-value function of the abnormal number recognition model based on reinforcement learning is implemented using an online random forest model. The sub-decision tree of the online random forest model can be updated and iterated through the following steps: Step S31, dividing the original node in the sub-decision tree into at least two sub-nodes based on a preset call feature division method; Step S32, based on the true values and predicted values corresponding to the call feature samples falling into the original node and falling into each sub-node, respectively calculate the mean square error of the original node and each sub-node; Step S33, calculating the information gain after node division based on the mean square error between the original node and each of the child nodes; Step S34: When the number of call feature samples falling into the original node is greater than the minimum number of split samples, and the information gain after node division is greater than the minimum split information gain, the original node is split, and based on the preset call feature division method, the call feature samples falling into the original node are allocated to child nodes.

[0054] Decision tree algorithms (including random forest algorithms) are a type of tree-based machine learning method based on data training and learning. These models offer excellent interpretability and can be used for classification problems in various fields. Based on the excellent interpretability of decision tree models and the excellent results of ensemble learning, this embodiment of the present invention uses a random forest model as the Q-value function model within the reinforcement learning framework for the abnormal number recognition model. Furthermore, by transforming the random forest model into an online random forest model, the model can be updated and iterated in real time based on environmental feedback during production.

[0055] For online learning, data is continuously generated. Random Forest can only continuously generate and optimize child decision trees as the data increases, rather than directly generating a complete random forest based on the training data, as in offline learning. Therefore, in an embodiment of the present invention, during the initialization phase of the online random forest model, T child decision trees with only root nodes can be generated, which form a random forest.

[0056] In this embodiment of the present invention, the optimization phase of the online random forest algorithm can split nodes in the child decision trees based on the continuous input of subsequent production data (i.e., call feature data for user numbers in future time slots). Because this embodiment of the present invention uses the online random forest algorithm to output continuous Q values, the mean squared error (MSE) can be used to characterize the quality of data partitioning and design information gain.

[0057] Specifically, when partitioning the original node in a child decision tree, specific features and feature values (i.e., a preset call feature partitioning method) can be selected to divide the original node into at least two child nodes. The mean squared error (MSE) of the original node is calculated based on the true and predicted values corresponding to the call feature samples falling within the original node. The mean squared error (MSE) of each child node is also calculated based on the true and predicted values corresponding to the call feature samples falling within each child node. Subsequently, the information gain after node partitioning can be calculated based on the MSE of the original node and each of its child nodes.

[0058] For example, assume that node t contains samples, the true values corresponding to these samples are , then the predicted value of the node is the mean of the true value of the sample, that is , then the mean square error of the node is When dividing a node, you can select specific features and eigenvalues to divide the node into two sub-nodes, and calculate the mean square error of the two sub-nodes respectively. and , the information gain after division is .

[0059] In the embodiment of the present invention, since the call feature data of the user number is continuously generated, the sample size and information gain falling into the node can be limited when the node is split.

[0060] Specifically, when the number of samples falling into the node is greater than the minimum number of split samples , and the information gain after node division is greater than the minimum split information gain , the node can be split and the call feature samples falling into the original node can be allocated to the corresponding child nodes based on the preset call feature division method.

[0061] By designing an external environment for model interaction and constructing an online random forest model, the embodiment of the present invention uses environmental feedback to modify and optimize the model in real time, enabling adaptive response to changing and abnormal situations. Specifically, by constructing an external environment for model interaction consisting of users, outbound call callbacks, and relevant departments, when the model decides to process or not process a number, the environment provides feedback on this action, enabling the online random forest model to automatically iterate and optimize.

[0062] In an optional embodiment, the number of times each sub-decision tree receives each arriving call feature sample is a target number, and the target number is a random number generated by a Poisson distribution.

[0063] In an embodiment of the present invention, in order to implement bagging in the case of real-time data stream, the data arrival mode of the call feature sample can be modeled as a Poisson distribution. , each child decision tree will receive each arriving sample k times (i.e., the target number of times), where k can be a Poisson distribution Generate a random number.

[0064] The embodiment of the present invention uses random numbers generated by Poisson distribution, so that each sub-decision tree only receives part of the data in the entire data set to achieve diversity, which is conducive to reducing overfitting.

[0065] In an optional embodiment, the sub-decision trees of the online random forest model can be optimized through the following steps: calculating the out-of-bag error of each sub-decision tree; for each sub-decision tree, if the out-of-bag error of the sub-decision tree is greater than the error upper limit and the total number of call feature samples received by the sub-decision tree is greater than the discardable upper limit, discard the sub-decision tree and initialize a new sub-decision tree.

[0066] The implementation of abnormal user behavior will continue to evolve as abnormal behavior identification methods are updated. Therefore, outdated abnormal behavior pattern knowledge will gradually become ineffective and harmful to the abnormal number recognition model. It is necessary to enable the model to forget this knowledge. Therefore, in the optimization phase of the online random forest model, the embodiment of the present invention designs a sub-decision tree elimination mechanism.

[0067] In an embodiment of the present invention, since bagging is used so that each sub-decision tree receives only part of the data in the entire data set to achieve diversity, the out-of-bag error (OOBE) can be used to characterize the performance of the sub-decision tree, thereby determining whether to retain the sub-decision tree based on the OOBE and age of the sub-decision tree.

[0068] When the OOBE of a sub-decision tree is greater than the upper error limit and the age of the sub-decision tree is greater than the upper limit that can be discarded, the sub-decision tree can be discarded and a new sub-decision tree can be initialized to maintain the scale of the random forest and learn new abnormal behavior characteristics. This allows the model to be adaptively upgraded and optimized over time to match new fraud characteristics, preventing the model from being overly affected by early historical samples.

[0069] In some embodiments, since the number of call feature samples received by a sub-decision tree increases with time, the number of samples seen by the sub-decision tree can be used to represent the age of the sub-decision tree.

[0070] It should be noted that the pre-training process of the abnormal number recognition model based on reinforcement learning provided in the embodiment of the present invention is similar to the automatic iterative update process after the model is put into online application. Therefore, the embodiment of the present invention will no longer describe the two processes separately.

[0071] In order to better understand the embodiment of the present invention, the embodiment of the present invention is described below through an embodiment.

[0072] Step 1: Obtain each user's call characteristics (such as call frequency) from user call record data, and obtain a pre-training dataset based on historical cases and outbound call return results.

[0073] The embodiment of the present invention can divide the user's call details into several time slices according to a specific time interval, and perform data aggregation analysis on the user's call data in each time slice to extract the characteristic statistical values therein, including but not limited to call frequency, called frequency, dispersion, average call interval duration, average call duration, connection rate, average off-hook ringing duration, called release rate, etc., and finally obtain the characteristic matrix of the user at time slice t . Where n represents the number of users, d represents the feature dimension, Represents the feature vector of the i-th user at time slice t.

[0074] Based on historical case information and the abnormal number database identified by the old model and confirmed by outbound call return, abnormal users with abnormal behavior risks within the research scope are obtained, that is, the user's label is obtained. ,in Indicates whether the i-th user is an abnormal user (0 indicates a normal user, 1 indicates an abnormal user).

[0075] Step 2: Build a reinforcement learning framework to help the abnormal number recognition model optimize parameters in real time based on environmental feedback and achieve a dynamic balance between timeliness and accuracy in identifying users with abnormal behavior risks.

[0076] The embodiment of the present invention can construct a time reward module, relying on the reinforcement learning framework to help the model consider timeliness factors and achieve a self-balance between "early recognition" and "accurate recognition". Specifically, in order to consider the impact of time factors on model recognition, a time reward module can be added to the reward function of the reinforcement learning framework to help the model learn the concept of time. As time goes by, the more data the model sees, the higher its accuracy will be, but the timeliness will be reduced accordingly. Therefore, when the model cannot determine whether the user number is an abnormal number in the current time slice and needs to observe the data characteristics of the number in the next time slice, the time reward module in the reward function can give the model a certain penalty to avoid the model ignoring timeliness in order to achieve accuracy. At the same time, due to the opposing relationship between accuracy and timeliness, a balance adjustment factor is also designed in the time reward module to adjust the penalty intensity of incorrect recognition and the penalty intensity of late recognition to achieve a self-balance between "early recognition" and "accurate recognition".

[0077] Step 3: Build an abnormal number recognition model based on online random forest. Its online design enables the model to be updated and iterated in real time based on environmental feedback.

[0078] The embodiment of the present invention can create an external environment for model interaction, and correct the online random forest model in real time through feedback from the external environment to achieve self-adaptation to variable fraud behaviors. Specifically, in order to improve the timeliness of model iteration, a model interaction environment composed of users, public security, outbound call personnel, etc. is designed. This environment can provide feedback on the model output, such as user complaints and restorations can be fed back to the model in real time to optimize the accuracy of the model, and public security case-related information can be fed back to the model in real time to optimize the model's "early identification" capabilities. At the same time, in order to avoid investing a lot of manpower in model iteration, the offline random forest model commonly used by current technical means is replaced with an online random forest model, so that the model can use real-time feedback data for iterative optimization by itself, reducing the degree of human intervention in the iterative process.

[0079] Step 4: Based on the pre-trained dataset, the abnormal number recognition model is iterated in a simulated production environment. Once the model meets the performance requirements in the offline training environment, the model is launched online. Based on online user feedback and user report information, the model parameters are updated in real time to adapt to changes in abnormal behavior.

[0080] In this embodiment of the present invention, the data in the pre-training dataset is arranged chronologically and fed into the model to simulate actual production input for pre-training. Once the model's accuracy (number of correctly identified numbers divided by the total number of numbers) and latency (the difference between the time an abnormal number is identified and the time the abnormal behavior begins) meet the requirements, the model is launched and subsequently iterated and upgraded in real time based on real-world feedback from the environment.

[0081] In summary, the embodiments of the present invention have the following technical advantages: (1) Constructing a time reward module to fill the gap in the timeliness of the existing abnormal number recognition model.

[0082] Current approaches to abnormal number recognition models primarily focus on identifying unusual numbers, without considering the speed of model recognition. This results in existing models having high coverage of numbers implicated in criminal activity, but often identifying them too late, leading to crimes. Ultimately, both late and no recognition can lead to users experiencing abnormal behavior, making the timeliness of model recognition a pressing issue.

[0083] To enable the model to quickly and promptly identify abnormal numbers, an embodiment of the present invention constructs a reinforcement learning framework and incorporates a time reward module into the framework's reward function. This module can impose a penalty when the model determines that subsequent time-slice features are needed to determine whether a number is abnormal, thereby reducing the probability of delayed recognition by the model. Furthermore, because "early recognition" and "accurate recognition" are often opposed to each other, a balance adjustment factor is constructed to adjust the penalty intensity for incorrect and late recognition, thereby achieving a self-balancing balance between "early recognition" and "accurate recognition."

[0084] (2) Construct a model interaction environment and online the random forest model to achieve adaptive optimization of the model for complex and changeable abnormal behavior situations.

[0085] Because abnormal call characteristics constantly change as both offensive and defensive techniques evolve, technicians must frequently optimize and iterate models based on unidentified or misidentified cases. However, this manual process is labor-intensive and model updates are often slow to keep pace with the rapid pace at which abnormal user groups, operating on an industrial scale, are updating their methods.

[0086] To improve the speed of model iteration, the embodiment of the present invention constructs a model interaction environment consisting of users, relevant departments (such as the public security department), outbound call personnel, etc., and onlineizes the random forest model. Through self-service interaction between the model and the environment, the model can learn new abnormal call characteristics without human intervention, thereby realizing real-time adaptive optimization of the model.

[0087] The abnormal number identification device provided by the present invention is described below. The abnormal number identification device described below and the abnormal number identification method described above can be referenced to each other.

[0088] Figure 3 Schematic diagram of the structure of the abnormal number identification device provided by the embodiment of the present invention. Figure 3 The embodiment of the present invention provides an abnormal number identification device, which may specifically include the following modules: The feature extraction module 310 is used to extract the call features of the user number in the current time slice from the user call record data; Anomaly identification module 320, configured to input the call features of the user number in the current time slice into an abnormal number identification model based on reinforcement learning to obtain an abnormal number identification result; Among them, the reward function of the abnormal number recognition model based on reinforcement learning includes a time reward module, which is used to give the model penalty feedback when the model determines whether the user number is an abnormal number based on the call characteristics of the user number in the future time slice.

[0089] The embodiment of the present invention extracts the call features of the user number in the current time slice from the user call record data, and inputs the call features of the user number in the current time slice into the abnormal number recognition model based on reinforcement learning to obtain the abnormal number recognition result. Therefore, in the abnormal number recognition process, by designing the abnormal number recognition model based on reinforcement learning and designing a time reward module in the reward function, when the model determines that it is necessary to judge whether the user number is an abnormal number based on the call features of the user number in the future time slice, a certain penalty feedback can be given to the model. Therefore, by constraining the timeliness of the model in identifying abnormal numbers, the probability of delayed recognition of the model is reduced, and the abnormal number can be identified in a timely and rapid manner, which is conducive to timely avoiding user losses.

[0090] Figure 4 An example of a physical structure diagram of an electronic device is shown below. Figure 4 As shown, the electronic device may include: a processor 410, a communications interface 420, a memory 430, and a communications bus 440. The processor 410, the communications interface 420, and the memory 430 communicate with each other via the communications bus 440. The processor 410 may invoke logic instructions in the memory 430 to execute an abnormal number identification method, which includes: extracting call characteristics of a user number in a current time slot from user call bill data; inputting the call characteristics of the user number in the current time slot into an abnormal number identification model based on reinforcement learning to obtain an abnormal number identification result; wherein the reward function of the abnormal number identification model based on reinforcement learning includes a time reward module for providing penalty feedback to the model when the model determines that a user number is an abnormal number based on the call characteristics of the user number in a future time slot.

[0091] Furthermore, the logic instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0092] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the abnormal number identification method provided by the above methods, which includes: extracting the call characteristics of the user number in the current time slice from the user call bill data; inputting the call characteristics of the user number in the current time slice into an abnormal number identification model based on reinforcement learning to obtain an abnormal number identification result; wherein the reward function of the abnormal number identification model based on reinforcement learning includes a time reward module, which is used to give the model penalty feedback when the model determines that it is necessary to determine whether the user number is an abnormal number based on the call characteristics of the user number in the future time slice.

[0093] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the abnormal number identification method provided by the above-mentioned methods, the method comprising: extracting the call characteristics of the user number in the current time slice from the user call bill data; inputting the call characteristics of the user number in the current time slice into an abnormal number identification model based on reinforcement learning to obtain an abnormal number identification result; wherein the reward function of the abnormal number identification model based on reinforcement learning includes a time reward module, which is used to give the model penalty feedback when the model determines that it is necessary to determine whether the user number is an abnormal number based on the call characteristics of the user number in a future time slice.

[0094] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0095] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for identifying abnormal numbers, characterized in that: include: Extract the call characteristics of the user number in the current time slice from the user call record data; Input the call features of the user number in the current time slice into the abnormal number recognition model based on reinforcement learning to obtain the abnormal number recognition result; Among them, the reward function of the abnormal number recognition model based on reinforcement learning includes a time reward module, which is used to give the model penalty feedback when the model determines whether the user number is an abnormal number based on the call characteristics of the user number in the future time slice.

2. The abnormal number identification method according to claim 1, characterized in that: The reward function is used to: If the model determines that the number of a normal user is a normal number and does not perform any action, or determines that the number of an abnormal user is an abnormal number and performs any action, a positive feedback of +1 is given to the model; If the model determines that the number of a normal user is an abnormal number and performs a treatment action, or determines that the number of an abnormal user is a normal number and does not perform a treatment action, a negative feedback of -1 is given to the model; In the case where the model needs to determine whether a user number is an abnormal number based on the call characteristics of the user number in the future time slice, and the decision is made late, the model is given penalty feedback.

3. The abnormal number identification method according to claim 1, characterized in that: Giving the model penalty feedback includes: Based on the number of consecutive determinations within a preset time period that the model determines based on future call characteristics whether the user number is an abnormal number, a preset function is used to calculate a first function value; the function value of the preset function increases as the number of consecutive determinations increases; Calculating a second function value using the preset function based on the number of time slices within the preset time period; Determining a delay penalty number that increases with the number of delays based on the first function value and the second function value; Penalty feedback is determined based on an adjustment factor for balancing timeliness and accuracy and the delay penalty number.

4. The abnormal number identification method according to claim 1, characterized in that: The abnormal number recognition model based on reinforcement learning includes an estimation model and a target model; After obtaining the abnormal number identification result, the method further includes: Executing an action corresponding to the abnormal number identification result on the user number; After receiving feedback from the external environment of the model interaction, the parameters of the estimation model are updated based on the target value output by the target model; the target value is obtained by inputting the call characteristics of the user number in the next time slice into the target model; After the estimation model is optimized and updated for a preset number of times, the parameters of the estimation model are assigned to the parameters of the target model to update the parameters of the target model.

5. The abnormal number identification method according to claim 1, characterized in that: The action-value function of the abnormal number recognition model based on reinforcement learning is implemented using an online random forest model. The sub-decision tree of the online random forest model is updated and iterated in the following way: Based on a preset call feature division method, the original node in the sub-decision tree is divided into at least two sub-nodes; Based on the true values and predicted values corresponding to the call feature samples falling into the original node and falling into each sub-node, respectively calculating the mean square error of the original node and each sub-node; Calculating the information gain after node division based on the mean square error between the original node and each of the child nodes; When the number of call feature samples falling into the original node is greater than the minimum number of split samples, and the information gain after node division is greater than the minimum split information gain, the original node is split, and based on the preset call feature division method, the call feature samples falling into the original node are allocated to child nodes.

6. The abnormal number identification method according to claim 5, characterized in that: The number of times each sub-decision tree receives each arriving call feature sample is the target number, and the target number is a random number generated by Poisson distribution.

7. The abnormal number identification method according to claim 6, characterized in that: The child decision trees of the online random forest model are optimized in the following way: Calculate the out-of-bag error of each sub-decision tree; For each sub-decision tree, if the out-of-bag error of the sub-decision tree is greater than the error upper limit and the age of the sub-decision tree is greater than the discard upper limit, the sub-decision tree is discarded and a new sub-decision tree is initialized.

8. An abnormal number identification device, characterized in that: include: The feature extraction module is used to extract the call features of the user number in the current time slice from the user call record data; An abnormality identification module, used to input the call characteristics of the user number in the current time slice into the abnormal number identification model based on reinforcement learning to obtain an abnormal number identification result; Among them, the reward function of the abnormal number recognition model based on reinforcement learning includes a time reward module, which is used to give the model penalty feedback when the model determines whether the user number is an abnormal number based on the call characteristics of the user number in the future time slice.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the abnormal number identification method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the abnormal number identification method according to any one of claims 1 to 7 is implemented.

11. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the abnormal number identification method according to any one of claims 1 to 7 is implemented.

Images