Abnormal number detection method and device, equipment, storage medium and program product
By integrating multi-source heterogeneous data through the Flink stream processing framework and dynamically calculating multi-dimensional anomaly indicators, the problem of insufficient real-time performance in detecting abnormal landline numbers is solved, enabling real-time and accurate detection of abnormal numbers.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-01-12
- Publication Date
- 2026-04-14
AI Technical Summary
In existing technologies, the detection of abnormal landline numbers is not real-time enough, static tag detection cannot respond to dynamic changes in real time, and offline algorithm models have long iteration cycles and weak real-time data processing capabilities, which cannot meet the real-time detection requirements.
By integrating multi-source heterogeneous data using the Flink stream processing framework, and through layered processing of real-time call detail records and offline customer tables, multi-dimensional anomaly indicators are dynamically calculated to achieve real-time detection of abnormal numbers in landline telephones.
It improves the real-time performance and accuracy of abnormal number detection, addresses the shortcomings of static tags and offline algorithm models, adapts to the dynamic and complex usage scenarios of landline phones, and addresses new fraud patterns.
Smart Images
Figure CN121864908A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of communication technology, and in particular to a method, apparatus, device, storage medium, and program product for detecting abnormal numbers. Background Technology
[0002] In the telecommunications industry, landline phones, as a traditional means of communication, offer advantages such as stable signal, clear call quality, traceable identity, and resistance to electromagnetic interference. However, with the widespread adoption of communication technologies, criminals use landlines to commit fraud, make harassing calls, and engage in other illegal activities. This harms users' rights and seriously impacts the reputation and business security of telecommunications operators.
[0003] In existing technologies, the detection of abnormal numbers in landline phones mostly relies on static labels or offline algorithm models. However, static label detection depends on offline data, cannot respond to dynamically changing call behavior in real time, and the label updates are lagging, making it difficult to cover new fraud patterns. On the other hand, offline algorithm models need to rely on large-scale historical data for training, resulting in long model iteration cycles and weak processing capabilities for real-time data, which cannot meet the needs of real-time detection.
[0004] Therefore, existing technologies suffer from insufficient real-time performance in detecting abnormal phone numbers. Summary of the Invention
[0005] This application provides a method, apparatus, device, storage medium, and program product for detecting abnormal numbers, in order to improve the real-time performance of abnormal number detection.
[0006] In a first aspect, embodiments of this application provide a method for detecting abnormal phone numbers, including:
[0007] Obtain the first real-time call detail record (CDR) and the offline customer table; wherein, the first real-time CDR is a dynamically updated streaming data set, including the first phone number; the offline customer table is a statically stored structured data set, including the second phone number and the customer name associated with the second phone number;
[0008] Match the first phone number with the second phone number;
[0009] If the first phone number and the second phone number are the same, then the customer name is assigned to the first real-time call detail record (CDR) to obtain the second real-time CDR.
[0010] Based on preset scenario keywords, filter the second real-time call detail records (CDRs) to obtain the third real-time CDRs;
[0011] Retrieve historical call detail records (CDRs); where historical CDRs are a collection of historical call records stored within a preset period.
[0012] Calculate multi-dimensional anomaly indicators based on real-time and historical call detail records;
[0013] If all the abnormal indicators in multiple dimensions meet the preset indicator threshold requirements, then the phone number is determined to be an abnormal number.
[0014] In one possible implementation, the multi-dimensional anomaly indicators include caller dispersion, the percentage of calls made within a first unit of time, the number of cities on the receiving end, the number of calls made within a second unit of time, the percentage of local calls made by the caller, the number of historical call days, and the peak number of average calls.
[0015] In one possible implementation, obtaining the first real-time call detail record (CDR) and offline customer table includes:
[0016] Retrieve the offline customer table and record the loading time of the offline customer table;
[0017] Obtain the first real-time call detail record (CDR) based on the preset delay duration.
[0018] In one possible implementation, obtaining the offline customer table includes:
[0019] Read the file path;
[0020] Retrieve the offline customer table under the file path.
[0021] In one possible implementation, after matching the first phone number with the second phone number, the process further includes:
[0022] If the first phone number and the second phone number are different, the streaming data corresponding to the first phone number will be stored in the intermediate state cache; the intermediate state cache is configured with a preset validity period.
[0023] In one possible implementation, a second real-time call detail record (CDR) is filtered based on preset scenario keywords to obtain a third real-time CDR, including:
[0024] Based on preset scenario keywords, the second real-time call detail record is filtered to obtain the first streaming data that does not include the preset scenario keywords and the second streaming data that includes the preset scenario keywords.
[0025] The first stream of data is stored in a pre-defined distributed file system;
[0026] Based on the second streaming data, a third real-time call detail record (CDR) is constructed.
[0027] In one possible implementation, after calculating multi-dimensional anomaly indicators based on the third real-time call detail records (CDRs) and historical CDRs, the method further includes:
[0028] If any of the multi-dimensional abnormal indicators does not meet any of the preset indicator threshold requirements, then the phone number is determined to be a normal number.
[0029] The streaming data corresponding to the normal numbers is stored in a pre-defined distributed file system.
[0030] In one possible implementation, after determining that a phone number is an abnormal number if all multi-dimensional anomaly indicators meet preset indicator threshold requirements, the method further includes:
[0031] The streaming data corresponding to the abnormal numbers is deduplicated to obtain the deduplicated streaming data.
[0032] The deduplicated streaming data is stored in a pre-defined database;
[0033] Calculate the number of anomalies based on the streaming data stored in the pre-set database;
[0034] If the number of anomalies exceeds the preset anomaly threshold, an alarm notification will be generated to prompt staff to perform manual maintenance.
[0035] Secondly, embodiments of this application provide an abnormal number detection device, comprising:
[0036] The first acquisition module is used to acquire the first real-time call detail record (CDR) and the offline customer table; wherein, the first real-time CDR is a dynamically updated streaming data set, including the first phone number; the offline customer table is a statically stored structured data set, including the second phone number and the customer name associated with the second phone number.
[0037] The matching module is used to match the first phone number with the second phone number.
[0038] The first module is used to assign the customer name to the first real-time call detail record (CDR) if the first phone number and the second phone number are the same, and then obtain the second real-time CDR.
[0039] The second module is used to filter the second real-time call detail records (CDRs) based on preset scenario keywords in order to obtain the third real-time CDRs.
[0040] The second acquisition module is used to acquire historical call detail records (CDRs); wherein, historical CDRs are a set of historical call records stored within a preset period.
[0041] The calculation module is used to calculate multi-dimensional anomaly indicators based on the third real-time call detail records and historical call detail records.
[0042] In one possible implementation, the multi-dimensional anomaly indicators include caller dispersion, the percentage of calls made within a first unit of time, the number of cities on the receiving end, the number of calls made within a second unit of time, the percentage of local calls made by the caller, the number of historical call days, and the peak number of average calls.
[0043] The determination module is used to determine a phone number as an abnormal number if all multi-dimensional abnormal indicators meet the preset indicator threshold requirements.
[0044] In one possible implementation, the multi-dimensional anomaly indicators include caller dispersion, the percentage of calls made within a first unit of time, the number of cities on the receiving end, the number of calls made within a second unit of time, the percentage of local calls made by the caller, the number of historical call days, and the peak number of average calls.
[0045] In one possible implementation, the first acquisition module may further be used for:
[0046] Retrieve the offline customer table and record the loading time of the offline customer table;
[0047] Obtain the first real-time call detail record (CDR) based on the preset delay duration.
[0048] In one possible implementation, obtaining the offline customer table includes:
[0049] Read the file path;
[0050] Retrieve the offline customer table under the file path.
[0051] In one possible implementation, the second obtaining module can also be used for:
[0052] Based on preset scenario keywords, the second real-time call detail record is filtered to obtain the first streaming data that does not include the preset scenario keywords and the second streaming data that includes the preset scenario keywords.
[0053] The first stream of data is stored in a pre-defined distributed file system;
[0054] Based on the second streaming data, a third real-time call detail record (CDR) is constructed.
[0055] In one possible implementation, after matching the first phone number with the second phone number, the abnormal number detection device further includes a storage module, wherein the storage module can specifically be used for:
[0056] If the first phone number and the second phone number are different, the streaming data corresponding to the first phone number will be stored in the intermediate state cache; the intermediate state cache is configured with a preset validity period.
[0057] In one possible implementation, the storage module can also be used for:
[0058] If any of the multi-dimensional abnormal indicators does not meet any of the preset indicator threshold requirements, then the phone number is determined to be a normal number.
[0059] The streaming data corresponding to the normal numbers is stored in a pre-defined distributed file system.
[0060] In one possible implementation, the storage module can also be used for:
[0061] The streaming data corresponding to the abnormal numbers is deduplicated to obtain the deduplicated streaming data.
[0062] The deduplicated streaming data is stored in a pre-defined database;
[0063] Calculate the number of anomalies based on the streaming data stored in the pre-set database;
[0064] If the number of anomalies exceeds the preset anomaly threshold, an alarm notification will be generated to prompt staff to perform manual maintenance.
[0065] Thirdly, embodiments of this application provide an abnormal number detection device, including: a memory and a processor;
[0066] The memory stores instructions that the computer executes;
[0067] The processor executes computer execution instructions stored in memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.
[0068] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.
[0069] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.
[0070] The abnormal number detection method, apparatus, device, storage medium, and program product provided in this application embodiment acquire a first real-time call detail record (CDR), an offline customer table, and historical CDRs; match a first phone number in the first real-time CDR with a second phone number in the offline customer table; if the first phone number and the second phone number match, assign the customer name from the offline customer table to the first real-time CDR to obtain a second real-time CDR; filter the second real-time CDR according to preset scenario keywords to obtain a third real-time CDR; calculate multi-dimensional abnormal indicators based on the third real-time CDR and historical CDRs; and if all multi-dimensional abnormal indicators meet preset indicator threshold requirements, the phone number is determined to be an abnormal number. Compared to existing technologies that rely solely on offline data to detect abnormal phone numbers, this method combines real-time CDRs, an offline customer table, and historical CDRs for processing and detection, ensuring the real-time nature and integrity of the data, thereby improving the real-time performance and accuracy of abnormal number detection. Attached Figure Description
[0071] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0072] Figure 1 A schematic diagram of an abnormal number detection system provided in this application;
[0073] Figure 2 Flowchart of the abnormal number detection method provided in this application Figure 1 ;
[0074] Figure 3 Flowchart of the abnormal number detection method provided in this application Figure 2 ;
[0075] Figure 4 Flowchart of the abnormal number detection method provided in this application Figure 3 ;
[0076] Figure 5 A schematic diagram of the structure of the abnormal number detection device provided in this application;
[0077] Figure 6 A schematic diagram of the structure of the abnormal number detection device provided in this application.
[0078] The accompanying drawings have illustrated specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to specific embodiments. Detailed Implementation
[0079] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.
[0080] It should be noted that all data involved in this application are information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant laws, regulations and standards, and corresponding operation portals are provided for users to choose to authorize or refuse.
[0081] Flink is an open-source stream processing framework with a core distributed stream data engine. It supports stateful computation of bounded or unbounded data streams and can be deployed on independent clusters, YARN (a resource manager), Kubernetes (an open-source container orchestration and management platform), and other modes. Flink on YARN is widely used by enterprises due to its advantages in resource management, fault tolerance, and compatibility with Hadoop (an open-source, scalable, and reliable distributed computing and storage framework).
[0082] The Flink runtime consists of two types of processes: JobManager and TaskManager. JobManager is responsible for scheduling tasks and coordinating fault tolerance; TaskManager runs as an independent JVM process and executes tasks through slots (the smallest unit of resources). A task can be split into multiple subtasks according to its degree of parallelism.
[0083] To address the issue of data loss caused by latency in multi-source heterogeneous data, Flink employs a timer mechanism: it registers processing / event timers based on KeyedProcessFunction, storing data in the state when not triggered, and triggering callback logic after a specified timestamp is reached.
[0084] In addition, Flink's side output stream can split the data stream: a single data stream can be split into multiple sub-data streams according to conditions. Each sub-stream can define different downstream processing logic and the data types can also be different. This can avoid copying data streams, process late data, and solve the problem that the Split operator cannot split the stream continuously. It is a commonly used stream processing strategy.
[0085] Landline phones, as a traditional means of communication, are widely used in business offices, homes, industry, specific sectors, and public services. However, the same number may be used temporarily by multiple users, resulting in complex call patterns. This leads to some landline numbers being exploited by criminals for fraud, harassment, and other illegal activities, harming user rights and impacting the reputation and business security of operators. Therefore, it is necessary to detect abnormal numbers in landline phones.
[0086] Existing methods for detecting abnormal phone numbers mainly fall into two categories: static analysis based on user tags and offline prediction based on algorithm models. Static analysis relies on offline data tag matching, which cannot respond to dynamic call behavior in real time, and tag updates are often delayed. Offline prediction requires large-scale training with historical data, resulting in long model iteration cycles and weak real-time processing capabilities. Because some abnormal phone number detection methods rely on offline or historical data, they are ill-suited to the dynamic and complex usage scenarios of landline phones, failing to meet the core requirements of real-time detection and struggling to address new fraud patterns.
[0087] To address the aforementioned issues, the core concept of this application is to integrate and dynamically analyze multi-source heterogeneous data based on the Flink stream processing framework to achieve real-time detection of abnormal numbers in landline telephones; and to dynamically calculate multi-dimensional anomaly indicators through layered processing of real-time call detail records and offline customer tables, thereby achieving real-time and accurate detection of abnormal numbers.
[0088] Optionally, Figure 1 This is a schematic diagram of an abnormal number detection system architecture provided in this application. Figure 1 As shown, the architecture of the abnormal number detection system includes at least one of a data acquisition device 101, a processing device 102, and a display device 103.
[0089] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the above architecture. In other feasible embodiments of this application, the above architecture may include more or fewer components than illustrated, or combine some components, or split some components, or arrange different components, which can be determined according to the actual application scenario and is not limited here. Figure 1 The components shown can be implemented in hardware, software, or a combination of both.
[0090] In the specific implementation process, the data acquisition device 101 may include an input / output interface or a communication interface. The data acquisition device 101 can be connected to the processing device through the input / output interface or the communication interface. The data acquisition device 101 can be used to obtain a first real-time call detail record (CDR) and an offline customer table, as well as to obtain historical CDRs. The first real-time CDR is a dynamically updated streaming data set, including a first phone number. The offline customer table is a statically stored structured data set, including a second phone number and the customer name associated with the second phone number. The historical CDR is a data set of historical call records stored within a preset period.
[0091] The processing device 102 can be used to match the first phone number with the second phone number; if the first phone number and the second phone number are the same, the customer name is assigned to the first real-time call detail record (CDR) to obtain the second real-time CDR; the second real-time CDR is filtered according to preset scenario keywords to obtain the third real-time CDR; multi-dimensional abnormal indicators are calculated based on the third real-time CDR and historical CDRs; if the multi-dimensional abnormal indicators all meet the preset indicator threshold requirements, the phone number is determined to be an abnormal number.
[0092] The display device 103 can also be a touch screen or the screen of a terminal device, used to receive user commands while displaying the above-mentioned content, so as to realize interaction with the user.
[0093] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.
[0094] Figure 2 Flowchart of the abnormal number detection method provided in this application Figure 1 ,like Figure 2 As shown, the method includes:
[0095] S201. Obtain the first real-time call detail record (CDR) and the offline customer table; wherein, the first real-time CDR is a dynamically updated streaming data set, including the first phone number; the offline customer table is a statically stored structured data set, including the second phone number and the customer name associated with the second phone number.
[0096] In this embodiment, the real-time call detail record (CDR) Kafka stream data of the landline is read and parsed as the first real-time CDR; wherein, the first real-time CDR includes the caller ID, number of calls, peer ID, call time and call location; the first phone number is the caller ID.
[0097] Read and parse Orc (Optimized Row Columnar) type offline files in HDFS (Hadoop Distributed File System) to create an offline customer table. The offline customer table includes the caller ID, number of calls, peer number, call time, call location, and customer name; the second phone number is the caller ID.
[0098] S202. Match the first phone number with the second phone number.
[0099] In this embodiment, under the Flink stream processing framework, the streaming data in the first real-time call detail record (CDR) is partitioned according to the first phone number. When the first real-time CDR arrives, it is first matched with the second offline customer table in the offline customer table based on the first phone number in the partitioned first real-time CDR. If the match is successful, step S203 is executed.
[0100] In some embodiments, after matching the first phone number with the second phone number, the method further includes:
[0101] If the first phone number and the second phone number are different, the streaming data corresponding to the first phone number will be stored in the intermediate state cache; the intermediate state cache is configured with a preset validity period.
[0102] In this embodiment, if the matching fails, the streaming data corresponding to the first phone number in the first real-time call detail record is stored in an intermediate state cache. For example, the intermediate state cache is configured with a preset validity period of 1 day. In some embodiments, before performing the matching process, a processing timer can be registered. For example, the processing timer can be set to 5 minutes. If the current time reaches the time set by the processing timer, the updated offline customer table in the distributed file system is synchronized in real time via broadcast stream, and the first phone number in the streaming data in the intermediate state cache is obtained and matched with the updated offline customer table until the matching is successful or the validity period of the intermediate state cache expires.
[0103] The abnormal number detection method provided in this application introduces the failed matching streaming data into a state caching mechanism and re-matches it using the updated offline customer table, thereby solving the matching omission problem caused by the delay of the offline customer table and improving the real-time performance and integrity of the data.
[0104] S203. If the first phone number is the same as the second phone number, then assign the customer name to the first real-time call detail record (CDR) to obtain the second real-time CDR.
[0105] In this embodiment, if the first phone number is the same as the second phone number, it indicates that there is a second phone number in the offline customer table that corresponds to the first phone number. The customer name associated with the second number is extracted from the offline customer table, and the customer name is assigned to the first real-time call detail record (CDR) to generate a second real-time CDR that includes the customer name.
[0106] S204. Based on preset scenario keywords, filter the second real-time call detail records (CDRs) to obtain the third real-time CDRs.
[0107] In some embodiments, filtering a second real-time call detail record (CDR) based on preset scenario keywords to obtain a third real-time CDR includes: filtering the second real-time CDR based on preset scenario keywords to obtain first streaming data that does not include the preset scenario keywords and second streaming data that includes the preset scenario keywords; storing the first streaming data in a preset distributed file system; and constructing the third real-time CDR based on the second streaming data.
[0108] In this embodiment, the use cases of landline telephones include enterprise offices, home life, industry, specific industries, or public service scenarios;
[0109] For example, if the use case of landline telephones is a specific industry (such as the hotel and accommodation industry), the preset scenario keywords include: hotel, inn, guesthouse, hostel or guesthouse.
[0110] By using preset scenario keywords, keyword matching is performed with customer names in each streaming data in the second real-time call detail record to obtain first streaming data that does not include preset scenario keywords and second streaming data that includes preset scenario keywords.
[0111] The system acquires first-stream data that does not include preset scenario keywords, performs format conversion on the first-stream data to obtain Orc format data, and outputs the Orc format data to a preset distributed file system for storage through a predefined side output stream.
[0112] Acquire second-stream data including preset scene keywords, and construct a third real-time call detail record.
[0113] S205. Obtain historical call detail records; wherein, historical call detail records are a set of historical call records stored within a preset period.
[0114] In this embodiment, historical call records of type Orc in the distributed file system are read and parsed as historical call detail records (CDRs). These historical CDRs are generated by an independent real-time fixed-line call detail record writing task. A data synchronization task is constructed using the Flink SQL (a structured query language built into a streaming computing framework) execution engine, subscribing to the real-time data stream of fixed-line call detail records in the Kafka message queue, loading and parsing the full field structure of the real-time data stream. Through the storage mapping configuration of Flink SQL, the parsed full-field CDR data is persistently stored in an Orc-formatted Hive (a data warehouse tool) data file using a streaming write method to obtain the historical CDRs. The historical CDRs are retained for six months and include historical normal numbers and historical abnormal numbers, as well as call behavior data corresponding to the historical normal and abnormal numbers. The call behavior data includes: caller ID, number of calls, peer ID, call time, and call location.
[0115] S206. Calculate multi-dimensional anomaly indicators based on the third real-time call detail record (CDR) and historical CDR.
[0116] In some embodiments, multi-dimensional anomaly indicators include caller dispersion, percentage of calls made within a first unit of time, number of cities on the other end, number of calls made within a second unit of time, percentage of local calls made by the caller, number of historical call days, and peak number of average calls.
[0117] In this embodiment, the calling dispersion refers to the ratio of the number of corresponding numbers of the calling number to the total number of calls made by the calling number within a statistical period; it is used to measure the degree of dispersion of the calling parties of the calling number. The higher the dispersion, the more diverse the calling parties are.
[0118] The first unit of time is 1 minute. The percentage of outgoing calls within the first unit of time refers to the ratio of the number of calls made by the outgoing number within 1 minute to the total number of calls made by this outgoing number.
[0119] The number of counterpart cities refers to the total number of different cities to which the counterpart number that made a call to the calling number belongs within the statistical period; it is used to measure the geographical coverage of the calling number's calls.
[0120] The second unit of time is 1 hour. The number of calls within the second unit of time refers to the number of calls made by the calling number per hour within the statistical period; it is used to detect the hourly call frequency of the calling number.
[0121] The local call percentage refers to the ratio of the number of local calls made by the calling number to the total number of calls made by that calling number; it is used to distinguish the proportion of local calls from out-of-town calls.
[0122] Historical call days refer to the total number of days within the statistical period of historical call details, during which the calling number has call records; it is used to reflect the historical activity period of the calling number.
[0123] Peak average number of calls refers to the maximum daily average number of calls made by a calling number within the statistical period of historical call records. It is used to exclude calling numbers that make normal, high-frequency calls by combining historical call behavior.
[0124] S207. If all the multi-dimensional abnormal indicators meet the preset indicator threshold requirements, then the phone number is determined to be an abnormal number.
[0125] In this embodiment, the preset indicator thresholds include a preset caller dispersion threshold, a preset caller call count percentage threshold, a preset peer city number threshold, a preset call count threshold, a preset caller local call count percentage threshold, a preset historical call days threshold, and a preset average call count threshold.
[0126] The preset threshold requirements include: if the calculated caller dispersion is greater than the preset caller dispersion threshold; if the calculated percentage of calls made within the first unit of time is less than the preset percentage of calls made; if the number of peer cities is greater than the preset number of peer cities threshold; if the number of calls made within the second unit of time is greater than the preset number of calls threshold; if the calculated percentage of local calls made is less than the preset percentage of local calls made; if the number of historical call days is greater than the preset number of historical call days threshold; and if the calculated peak value of the average number of calls is greater than the preset average number of calls threshold.
[0127] If multiple abnormal indicators simultaneously meet the preset indicator threshold requirements, the phone number is determined to be an abnormal number.
[0128] In some embodiments, after calculating multi-dimensional anomaly indicators based on third real-time call detail records (CDRs) and historical CDRs, the method further includes:
[0129] If any of the multi-dimensional abnormal indicators does not meet any of the preset indicator threshold requirements, the phone number is determined to be a normal number; the streaming data corresponding to the normal number is stored in the preset distributed file system.
[0130] In this embodiment, if the phone number is determined to be a normal number, the streaming data corresponding to the normal number is output to a preset distributed file system for storage through a predefined side output stream.
[0131] The abnormal number detection method provided in this application obtains a first real-time call detail record (CDR), an offline customer table, and historical CDRs. It then matches a first phone number in the first real-time CDR with a second phone number in the offline customer table. If the first and second phone numbers match, the customer name from the offline customer table is assigned to the first real-time CDR to obtain a second real-time CDR. Based on preset scenario keywords, the second real-time CDR is filtered to obtain a third real-time CDR. Multi-dimensional anomaly indicators are calculated based on the third real-time CDR and historical CDRs. These multi-dimensional anomaly indicators are then compared; if all indicators meet preset threshold requirements, the phone number is determined to be an abnormal number. By combining real-time CDRs and the offline customer table, the method solves the problem of time-series inconsistency between real-time streaming data and offline data, reduces the probability of data loss, and ensures the real-time performance and integrity of the data. Furthermore, the combination of multi-dimensional anomaly indicators for abnormal number detection improves the real-time performance and accuracy of abnormal number detection.
[0132] Figure 3 Flowchart of the abnormal number detection method provided in this application Figure 2 ,like Figure 3 As shown, in this embodiment... Figure 2 Based on the embodiments, the method of obtaining the first real-time call detail record and offline customer table in step S201 above will be described in detail. The method includes:
[0133] S301, Read file path.
[0134] In this embodiment, the file path to be read includes the name of the distributed file system database where the offline customer table is located, the table name, date information, and the file name.
[0135] In response to the request to read the offline customer table, the target date information of the offline customer table to be read is obtained and matched with the date information in the file path to obtain the file path corresponding to the target time information.
[0136] Furthermore, the offline customer table updated under this file path is monitored in real time. The parallelism of path reading is set to 1 to ensure the uniqueness and timing accuracy of file path monitoring, avoid multiple parallel threads reading at the same time, which would lead to duplicate capture of path update information or disordered timing, and improve the accuracy of data.
[0137] S302. Based on the file path, obtain the offline customer table under the file path.
[0138] In this embodiment, when an update to the offline customer table under the file path is detected, a data loading process is triggered to obtain the offline customer table under the file path, so as to ensure that the timeliness of the offline customer table matches that of the first real-time call detail record.
[0139] Furthermore, parallel resources are dynamically allocated based on the amount of data in the offline customer table under the file path to solve the problem of long processing time caused by single-threaded reading of large amounts of data in the offline customer table, thereby improving the reading speed of the offline customer table under the file path.
[0140] Optionally, once the offline customer table has finished loading, the loading time of the offline customer table can be recorded.
[0141] S303. Obtain the first real-time call detail record (CDR) based on the preset delay duration.
[0142] In this embodiment, the preset delay duration can be dynamically set based on the historical loading duration of the offline customer table.
[0143] For example, a global tag variable Flag is set with a default boolean value of True. If the boolean value of the global tag variable Flag is read as True, the thread that obtains the first real-time call detail record will be set to sleep for 5 minutes according to the preset delay duration, and the boolean value of the global tag variable Flag will be changed to False.
[0144] The abnormal number detection method provided in this application solves the problems of loading delay of offline customer table and out-of-order real-time streaming data by introducing a two-step loading method of first reading the file path and then reading the offline customer table under the file path, and a thread sleep mechanism. This improves the integrity and accuracy of multi-source heterogeneous data, thereby ensuring the accuracy of subsequent abnormal number detection.
[0145] Figure 4 Flowchart of the abnormal number detection method provided in this application Figure 3 ,like Figure 4 As shown, in this embodiment... Figure 2 Based on the embodiments, after determining the phone number as an abnormal number in step S207 if all multi-dimensional abnormal indicators meet the preset indicator threshold requirements, the method further includes:
[0146] S401. Perform deduplication on the streaming data corresponding to the abnormal numbers to obtain the deduplicated streaming data.
[0147] In this embodiment, the deduplication process includes memory deduplication and storage deduplication.
[0148] The streaming data corresponding to the abnormal number includes a distribution identifier, which includes "distributed" and "not distributed". The initial value of the distribution identifier is "not distributed". If the distribution identifier in the streaming data corresponding to the abnormal number is "not distributed", then storage deduplication is performed and the distribution identifier is updated to "distributed". If the distribution identifier in the streaming data corresponding to the abnormal number is "distributed", then the streaming data corresponding to this abnormal number is directly filtered.
[0149] The streaming data corresponding to the abnormal numbers deduplicated in memory is written to a preset database using a structured query statement that includes the Ignore keyword. In the preset database, a unique index is created for the abnormal number field to achieve in-memory deduplication of the streaming data corresponding to the abnormal numbers.
[0150] S402. Store the deduplicated streaming data into a preset database.
[0151] In this embodiment, if the streaming data corresponding to the abnormal number is deduplicated in memory and in storage, the deduplicated streaming data can be stored in a preset database.
[0152] S403. Calculate the number of anomalies based on the streaming data stored in the preset database.
[0153] In this embodiment, for example, if the statistical period is daily, the number of streaming data stored in the preset database within that day is counted as the number of anomalies.
[0154] S404. If the number of anomalies exceeds the preset anomaly threshold, an alarm notification will be generated to prompt staff to perform manual maintenance.
[0155] In this embodiment, the preset anomaly threshold is calculated based on the average number of historical anomalies and a preset overflow amount. The formula for calculating the preset anomaly threshold is as follows:
[0156] The preset anomaly threshold = the average number of historical anomalies × (100% + preset overflow amount).
[0157] For example, if the historical average number of anomalies is the average number of anomalies from the previous week, and the number is 100, and the preset overflow limit is 10%, then the preset anomaly threshold for the day is 110. If the number of anomalies on the day is 120, an alarm notification will be generated to prompt staff to perform manual maintenance and stop the detection of anomalies.
[0158] The abnormal number detection method provided in this application ensures the accuracy and real-time performance of abnormal number detection through a dual deduplication mechanism of memory deduplication and storage deduplication, and introduces a circuit breaker mechanism to achieve automated prediction of task stability, thereby reducing manual operation and maintenance costs.
[0159] Figure 5 A schematic diagram of the structure of the abnormal number detection device provided in this application is shown below. Figure 5 As shown, the abnormal number detection device provided in this embodiment includes:
[0160] The first acquisition module 501 is used to acquire a first real-time call detail record (CDR) and an offline customer table; wherein, the first real-time CDR is a dynamically updated streaming data set, including a first phone number; the offline customer table is a statically stored structured data set, including a second phone number and a customer name associated with the second phone number.
[0161] The matching module 502 is used to match the first phone number with the second phone number.
[0162] The first module 503 is used to assign the customer name to the first real-time call detail record (CDR) if the first phone number and the second phone number are the same, and then obtain the second real-time CDR.
[0163] The second module 504 is used to filter the second real-time call detail records (CDRs) based on preset scenario keywords to obtain the third real-time CDRs.
[0164] The second acquisition module 505 is used to acquire historical call detail records (CDRs); wherein, historical CDRs are a set of historical call records stored within a preset period.
[0165] The calculation module 506 is used to calculate multi-dimensional anomaly indicators based on the third real-time call detail records and historical call detail records.
[0166] In one possible implementation, the multi-dimensional anomaly indicators include caller dispersion, the percentage of calls made within a first unit of time, the number of cities on the receiving end, the number of calls made within a second unit of time, the percentage of local calls made by the caller, the number of historical call days, and the peak number of average calls.
[0167] The determination module 507 is used to determine the phone number as an abnormal number if all multi-dimensional abnormal indicators meet the preset indicator threshold requirements.
[0168] In one possible implementation, the first acquisition module 501 may further be used for:
[0169] Retrieve the offline customer table and record the loading time of the offline customer table;
[0170] Obtain the first real-time call detail record (CDR) based on the preset delay duration.
[0171] In one possible implementation, obtaining the offline customer table includes:
[0172] Read the file path;
[0173] Retrieve the offline customer table under the file path.
[0174] In one possible implementation, the second obtaining module 504 can also be used for:
[0175] Based on preset scenario keywords, the second real-time call detail record is filtered to obtain the first streaming data that does not include the preset scenario keywords and the second streaming data that includes the preset scenario keywords.
[0176] The first stream of data is stored in a pre-defined distributed file system;
[0177] Based on the second streaming data, a third real-time call detail record (CDR) is constructed.
[0178] In one possible implementation, after matching the first phone number with the second phone number, the abnormal number detection device further includes a storage module, wherein the storage module can specifically be used for:
[0179] If the first phone number and the second phone number are different, the streaming data corresponding to the first phone number will be stored in the intermediate state cache; the intermediate state cache is configured with a preset validity period.
[0180] In one possible implementation, the storage module can also be used for:
[0181] If any of the multi-dimensional abnormal indicators does not meet any of the preset indicator threshold requirements, then the phone number is determined to be a normal number.
[0182] The streaming data corresponding to the normal numbers is stored in a pre-defined distributed file system.
[0183] In one possible implementation, the storage module can also be used for:
[0184] The streaming data corresponding to the abnormal numbers is deduplicated to obtain the deduplicated streaming data.
[0185] The deduplicated streaming data is stored in a pre-defined database;
[0186] Calculate the number of anomalies based on the streaming data stored in the pre-set database;
[0187] If the number of anomalies exceeds the preset anomaly threshold, an alarm notification will be generated to prompt staff to perform manual maintenance.
[0188] The abnormal number detection device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.
[0189] Figure 6 This is a schematic diagram of the structure of the abnormal number detection device provided in this application. Figure 6As shown, the abnormal number detection device provided in this embodiment includes at least one processor 601 and a memory 602. Optionally, the abnormal number detection device further includes a communication component 603. The processor 601, memory 602, and communication component 603 are connected via a bus 604.
[0190] In a specific implementation, at least one processor 601 executes computer execution instructions stored in memory 602, causing at least one processor 601 to perform the above-described method.
[0191] The specific implementation process of processor 601 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.
[0192] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.
[0193] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.
[0194] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.
[0195] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.
[0196] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.
[0197] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.
[0198] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.
[0199] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.
[0200] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0201] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.
[0202] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0203] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.
[0204] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.
Claims
1. A method for detecting abnormal phone numbers, characterized in that, include: Obtain a first real-time call detail record (CDR) and an offline customer table; wherein, the first real-time CDR is a dynamically updated streaming data set, including a first phone number; the offline customer table is a statically stored structured data set, including a second phone number and a customer name associated with the second phone number; The first phone number is matched with the second phone number. If the first phone number is the same as the second phone number, then the customer name is assigned to the first real-time call detail record (CDR) to obtain the second real-time CDR. Based on preset scenario keywords, the second real-time call detail record (CDR) is filtered to obtain the third real-time CDR; Obtain historical call detail records; wherein, the historical call detail records are a data set of historical call records stored within a preset period; Calculate multi-dimensional anomaly indicators based on the third real-time call detail record (CDR) and the historical CDR. If all the multi-dimensional anomaly indicators meet the preset indicator threshold requirements, then the phone number is determined to be an abnormal number.
2. The method according to claim 1, characterized in that, The multi-dimensional anomaly indicators include caller dispersion, the percentage of calls made within the first unit of time, the number of cities on the other end, the number of calls made within the second unit of time, the percentage of local calls made by the caller, the number of historical call days, and the peak number of average calls.
3. The method according to claim 1, characterized in that, The process of obtaining the first real-time call detail record (CDR) and offline customer table includes: Obtain the offline customer table and record the loading time of the offline customer table; The first real-time call detail record (CDR) is obtained based on the preset delay duration.
4. The method according to claim 3, characterized in that, The step of obtaining the offline customer table includes: Read the file path; Based on the file path, obtain the offline customer table under the file path.
5. The method according to claim 1, characterized in that, After the matching process between the first phone number and the second phone number, the method further includes: If the first phone number is different from the second phone number, the streaming data corresponding to the first phone number is stored in an intermediate state cache; wherein, the intermediate state cache is configured with a preset validity period.
6. The method according to claim 1, characterized in that, Based on preset scenario keywords, the second real-time call detail records (CDRs) are filtered to obtain the third real-time CDRs, including: Based on the preset scene keywords, the second real-time call detail record is filtered to obtain first streaming data that does not include the preset scene keywords and second streaming data that includes the preset scene keywords; The first stream of data is stored in a pre-defined distributed file system; Based on the second streaming data, a third real-time call detail record (CDR) is constructed.
7. The method according to claim 1, characterized in that, After calculating the multi-dimensional anomaly indicators based on the third real-time call detail record (CDR) and the historical CDR, the method further includes: If the multi-dimensional anomaly indicators do not meet any preset indicator threshold requirements, then the phone number is determined to be a normal number; The streaming data corresponding to the normal number is stored in the preset distributed file system.
8. The method according to claim 1, characterized in that, After determining that the phone number is an abnormal number if the multi-dimensional anomaly indicators meet the preset indicator thresholds, the method further includes: The streaming data corresponding to the abnormal numbers is deduplicated to obtain the deduplicated streaming data. The deduplicated streaming data is stored in a preset database; Calculate the number of anomalies based on the streaming data stored in the preset database; If the number of anomalies exceeds a preset anomaly threshold, an alarm notification is generated to prompt staff to perform manual maintenance.
9. A device for detecting abnormal phone numbers, characterized in that, include: The first acquisition module is used to acquire a first real-time call detail record (CDR) and an offline customer table; wherein, the first real-time CDR is a dynamically updated streaming data set, including a first phone number; the offline customer table is a statically stored structured data set, including a second phone number and a customer name associated with the second phone number; A matching module is used to match the first phone number with the second phone number; The first obtaining module is used to assign the customer name to the first real-time call detail record (CDR) if the first phone number is the same as the second phone number, thereby obtaining the second real-time CDR. The second obtaining module is used to filter the second real-time call detail record (CDR) based on preset scenario keywords to obtain the third real-time CDR. The second acquisition module is used to acquire historical call detail records (CDRs); wherein, the historical CDRs are a data set of historical third real-time CDRs stored within a preset period. The calculation module is used to calculate multi-dimensional anomaly indicators based on the third real-time call detail record (CDR) and the historical CDR. The determination module is used to determine the phone number as an abnormal number if the multi-dimensional abnormal indicators meet a preset indicator threshold.
10. A device for detecting abnormal phone numbers, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-8.
11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-8.
12. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method described in any one of claims 1-8.