Charging deduplication method, apparatus, device, storage medium, and program product

Abstract

This invention provides a billing deduplication method, apparatus, device, storage medium, and program product. The method includes: in response to a billing deduplication request, obtaining the call detail records (CDRs) to be deduplicated corresponding to the request; determining that the CDR to be deduplicated is an offline CDR, and performing billing deduplication on the offline CDR based on its deduplication key value; determining that the CDR to be deduplicated is an online CDR, and determining the message type of the online CDR based on its deduplication key value; and performing billing deduplication on the online CDR based on the deduplication service processing mechanism corresponding to the message type of the online CDR. This invention avoids duplicate billing and ensures more accurate billing by differentiating and processing different types of CDRs. For offline CDRs, billing deduplication is performed based on the deduplication key value; for online CDRs, the appropriate deduplication service processing mechanism is selected based on the message type.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a billing and deduplication method, apparatus, equipment, storage medium, and program product. Background Technology

[0002] With the development of network technology and the continuous enrichment of internet applications, data traffic consumption is growing rapidly and user demands are becoming increasingly active, posing new challenges to operators while also accelerating the rapid iteration of communication technologies. Therefore, billing deduplication is of paramount importance.

[0003] Current billing deduplication relies on a physical database. It defines deduplication keys for each call detail record (CDR) and stores these keys in a physical database deduplication key-value table. During the sorting process, this table is read for deduplication. However, this method depends on an Oracle physical database deduplication table, and as the volume of CDRs increases, the efficiency of the deduplication process decreases.

[0004] Therefore, there is an urgent need to propose a billing and deduplication method, device, equipment, storage medium, and program product to solve the above-mentioned technical problems. Summary of the Invention

[0005] This invention provides a billing deduplication method, apparatus, device, storage medium, and program product to solve the problem that the existing technology relies on Oracle physical database deduplication tables to achieve billing deduplication, which leads to a decrease in the deduplication process efficiency as the volume of data service call orders increases.

[0006] This invention provides a billing deduplication method, comprising:

[0007] In response to a billing deduplication request, obtain the call detail records (CDRs) to be deduplicated corresponding to the billing deduplication request;

[0008] If the call detail record to be deduplicated is determined to be an offline call detail record, billing deduplication is performed on the offline call detail record based on the deduplication key value of the offline call detail record;

[0009] If the call detail record to be deduplicated is determined to be an online call detail record, the message type of the online call detail record is determined based on the deduplication key value of the online call detail record;

[0010] Based on the deduplication service processing mechanism corresponding to the message type of the online call detail record (CDR), the online CDR is deduplicated for billing purposes.

[0011] According to a billing deduplication method provided by the present invention, the step of performing billing deduplication on the offline call detail records (CDRs) based on the deduplication key value of the offline CDRs includes:

[0012] The deduplication key value of the offline call detail record (CDR) is checked for duplicates against the deduplication key values ​​of all online CDRs.

[0013] If the deduplication key values ​​of the offline call detail records are not duplicated, the billing deduplication is performed on the offline call detail records based on the deduplication results of the call detail record key values.

[0014] If the deduplication key value of the offline call detail record (CDR) is duplicated, the offline CDR is marked with a duplicate error code, which indicates that the offline CDR does not need to be billed.

[0015] According to a billing deduplication method provided by the present invention, determining that the call detail record (CDR) to be deduplicated is an online CDR, and determining the message type of the online CDR based on the deduplication key value of the online CDR, includes:

[0016] If the time slice in the deduplication key value of the online call detail record is greater than or equal to the difference between the current time and the duration of the hotspot, the message type of the online call detail record is determined to be data within the hotspot.

[0017] If the time slice in the deduplication key value of the online call detail record (CDR) is less than or equal to the difference between the current time and the duration of the hotspot, the message type of the online CDR is determined to be data outside the hotspot.

[0018] According to a billing deduplication method provided by the present invention, the deduplication service processing mechanism based on the message type corresponding to the online call detail record (CDR) performs billing deduplication on the online CDR, including:

[0019] When the message type of the online call detail record (CDR) is data within a hotspot, determine whether the time offset between the maximum time point of the time slice in the deduplication key value of the online CDR and the current time exceeds the Redis Hash hotspot duration;

[0020] If the time offset does not exceed the Redis Hash hotspot duration, the deduplication key values ​​of the online call detail records are checked using a Bloom filter;

[0021] Based on the deduplication results from the Bloom filter, the online call detail records are deduplicated for billing purposes.

[0022] According to a billing deduplication method provided by the present invention, the method further includes:

[0023] If the time offset exceeds the Redis Hash hotspot duration, determine whether the time offset exceeds the Bloom filter hotspot duration.

[0024] If the time offset does not exceed the hotspot duration of the Bloom filter, the online call detail records are deduplicated for billing based on the Bloom filter and the RedisHash table.

[0025] According to a billing deduplication method provided by the present invention, the method further includes:

[0026] If the time offset exceeds the hotspot duration of the Bloom filter, billing deduplication is performed on the online call detail records based on the Redis hash table and the HBase table.

[0027] The present invention also provides a billing and de-weighting device, comprising:

[0028] The first billing and deduplication module is used to respond to a billing and deduplication request and obtain the call detail records to be deduplicated corresponding to the billing and deduplication request.

[0029] The second billing and deduplication module is used to perform billing and deduplication on the offline call detail record (CDR) based on the deduplication key value of the offline CDR when it is determined that the CDR to be deduplicated is an offline CDR.

[0030] The third billing and deduplication module is used to determine the message type of the online call detail record (CDR) based on the deduplication key value of the online CDR when it is determined that the CDR to be deduplicated is an online CDR.

[0031] The fourth billing and deduplication module is used to perform billing and deduplication on the online call detail records (CDRs) based on the deduplication service processing mechanism corresponding to the message type of the online CDR.

[0032] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the billing and deduplication method described above.

[0033] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the billing and deduplication method as described above.

[0034] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the billing and deduplication method described above.

[0035] The billing deduplication method, apparatus, device, storage medium, and program product provided by this invention, in response to a billing deduplication request, obtains the call detail records (CDRs) to be deduplicated corresponding to the request; determines that the CDR to be deduplicated is an offline CDR, and performs billing deduplication on the offline CDR based on its deduplication key value; determines that the CDR to be deduplicated is an online CDR, and determines the message type of the online CDR based on its deduplication key value; and performs billing deduplication on the online CDR for the online message based on the deduplication service processing mechanism corresponding to the message type of the online CDR. This invention, by distinguishing and processing different types of CDRs (offline and online CDRs), can ensure a more accurate billing process. For offline CDRs, deduplication billing based on the deduplication key value can avoid duplicate billing; for online CDRs, selecting the appropriate deduplication service processing mechanism based on the message type further ensures the accuracy of billing. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0037] Figure 1 This is one of the flowcharts of the billing and deduplication method provided by the present invention;

[0038] Figure 2 This is the second flowchart of the billing and deduplication method provided by the present invention;

[0039] Figure 3 This is a schematic diagram of the billing and de-weighting device provided by the present invention;

[0040] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0042] Figure 1 This is one of the flowcharts illustrating the billing and deduplication method provided by the present invention; such as Figure 1 As shown, the method includes:

[0043] Step 110: In response to the billing deduplication request, obtain the call detail record type corresponding to the billing deduplication request;

[0044] A billing deduplication request typically indicates that a user wants the billing deduplication system to deduplicate the submitted call detail records (CDRs) to avoid duplicate billing or data redundancy. A billing deduplication request can include one or multiple CDRs to be deduplicated.

[0045] In this step, after receiving the billing deduplication request, the specific call detail record (CDR) type to be deduplicated carried in the billing deduplication request is determined.

[0046] Call detail record (CDR) type typically refers to the type of service represented by the call detail record. In practice, there may be multiple CDR types, such as offline CDRs and online CDRs. Each CDR type may have a different format, contain different information, and may require different deduplication strategies.

[0047] Step 120: Determine that the call detail record to be deduplicated is an offline call detail record, and perform billing deduplication on the offline call detail record according to the deduplication key value of the offline call detail record;

[0048] Offline call detail records (CDRs) refer to a series of CDR records that are independently saved to a separate offline file by network element devices for subsequent processing during the online billing process due to certain abnormal situations (such as message retransmission failure or message abnormality) that prevent online messages from being processed normally.

[0049] Deduplication keys are a set of information used to determine whether call detail records (CDRs) are duplicated. These keys typically include key information such as user ID, timestamp, and service type. They ensure that each CDR is unique within the billing and deduplication system.

[0050] In this embodiment, for offline call detail records (CDRs) with abnormal messages, the billing and deduplication system will check their deduplication key values ​​to identify and remove duplicate or similar CDRs.

[0051] Step 130: Determine that the call detail record to be deduplicated is an online call detail record, and determine the message type of the online call detail record based on the deduplication key value of the online call detail record;

[0052] In this embodiment, when the call detail record (CDR) type is online CDR, the system determines the message type of the online CDR based on its deduplication key value. The message types of online CDRs are mainly divided into hotspot data and non-hotspot data.

[0053] Alternatively, the online call detail record (CDR) can be determined as either hotspot data or non-hotspot data by using the time slicing in the deduplication key value.

[0054] Among them, time sharding in deduplication key values ​​refers to sharding or dividing data according to time, which is used for more detailed management and classification of data in deduplication processing.

[0055] Specifically, the process begins by parsing the time field in the deduplication key and converting it into time-sliced ​​data. Time slices can be divided at different granularities, such as by hour, day, week, or month. Then, based on the time slices in the deduplication key and the preset hot / cold data segmentation rules, the system determines whether the online call detail record (CDR) falls into the hotspot category or the non-hotspot category.

[0056] Step 140: Based on the deduplication service processing mechanism corresponding to the message type of the online call detail record (CDR), perform billing deduplication on the online CDR.

[0057] In this embodiment, the deduplication service mechanism to be used is determined based on the message type of the online call detail record (CDR). The message type can be data within a hotspot or data outside a hotspot.

[0058] For example, online call detail records (CDRs) within hotspot data, due to their high access volume and real-time requirements, typically employ more efficient deduplication mechanisms. These mechanisms include, but are not limited to, using caching techniques or distributed processing. By optimizing the deduplication process, the system can ensure that online CDRs from hotspot data are processed promptly.

[0059] For example, for online call detail records (CDRs) that are not within hotspots, conventional deduplication mechanisms, such as deduplication algorithms based on database queries, can be used to deduplicatize these CDRs.

[0060] Based on the message type deduplication service mechanism corresponding to online call detail records (CDRs), the deduplication process can be flexibly selected and optimized according to the characteristics of different types of data and business needs, thereby achieving efficient and accurate billing deduplication of online CDRs for online messages.

[0061] This application embodiment, in response to a billing and deduplication request, obtains the call detail records (CDRs) to be deduplicated corresponding to the request. If the CDR to be deduplicated is an offline CDR, billing and deduplication are performed on the offline CDR based on its deduplication key value. If the CDR to be deduplicated is an online CDR, the message type of the online CDR is determined based on its deduplication key value. Based on the deduplication service processing mechanism corresponding to the message type of the online CDR, billing and deduplication are performed on the online CDRs containing online messages. This invention, by distinguishing and processing different types of CDRs (offline and online CDRs), ensures a more accurate billing process. For offline CDRs, deduplication and billing based on the deduplication key value avoids duplicate billing; for online CDRs, selecting the appropriate deduplication service processing mechanism based on the message type further ensures the accuracy of billing.

[0062] In some embodiments, the step of deduplicating the offline call records for billing purposes based on the deduplication key value includes:

[0063] The deduplication key value of the offline call detail record (CDR) is checked for duplicates against the deduplication key values ​​of all online CDRs.

[0064] If the deduplication key values ​​of the offline call detail records are not duplicated, the billing deduplication is performed on the offline call detail records based on the deduplication results of the call detail record key values.

[0065] If the deduplication key value of the offline call detail record (CDR) is duplicated, the offline CDR is marked with a duplicate error code, which indicates that the offline CDR does not need to be billed.

[0066] First, for offline call detail records (CDRs) corresponding to abnormal messages, the deduplication key values ​​for these offline CDRs are retrieved. Next, the retrieved deduplication key values ​​for offline CDRs are compared with the deduplication key values ​​for all online CDRs to check for duplicates.

[0067] If the deduplication key value of an offline call detail record (CDR) is not repeated, meaning it does not match the deduplication key value of any online CDR, it indicates that the online message corresponding to that offline CDR has not been billed before.

[0068] In this case, further deduplication is performed based on the call detail record (CDR) key values ​​from the offline CDR. The CDR key value is an identifier within the CDR used to further distinguish different CDRs. After deduplication by CDR key values, the system will process the billing for the offline CDR, ensuring that duplicate billing does not occur due to duplicate CDRs.

[0069] If the deduplication key value of an offline call detail record (CDR) is duplicated—that is, it is identical to the deduplication key value of one or more online CDRs—it indicates that the online message corresponding to the offline CDR has already been billed online. To avoid duplicate billing, the system will mark the offline CDR with a duplicate error code. This duplicate error code is a specific marker used to indicate that this offline CDR does not need to be billed due to duplication.

[0070] This application's embodiments, through the above deduplication process, can accurately distinguish which offline call detail records (CDRs) have not been processed and require billing, and which offline CDRs are duplicates that have already been processed and do not require re-billing. This improves the accuracy and efficiency of billing and avoids duplicate billing. Furthermore, error code identification facilitates subsequent problem tracking and handling.

[0071] In some embodiments, determining that the call detail record (CDR) to be deduplicated is an online CDR, and determining the message type of the online CDR based on the deduplication key value of the online CDR, includes:

[0072] If the time slice in the deduplication key value of the online call detail record is greater than or equal to the difference between the current time and the duration of the hotspot, the message type of the online call detail record is determined to be data within the hotspot.

[0073] If the time slice in the deduplication key value of the online call detail record (CDR) is less than or equal to the difference between the current time and the duration of the hotspot, the message type of the online CDR is determined to be data outside the hotspot.

[0074] In this embodiment, the message type of the online call detail record (CDR) is determined based on the following three parameters:

[0075] Time slice t: Time information in the online call detail record (CDR) deduplication key value, representing the time period during which the CDR occurred.

[0076] Hotspot duration d: This is a preset value used to distinguish hotspot time periods.

[0077] Current time s: This is the current time the system is processing this online call detail record.

[0078] Specifically, the deduplication keys for online call detail records are usually stored in a Redis cluster. The first value of each deduplication key in the Redis cluster is used as a time shard, which represents the time attribute of the data or message corresponding to this deduplication key.

[0079] In this embodiment, if the time segment t is greater than or equal to the difference between the current time s and the hotspot duration d (i.e., t>=(sd)), then it means that the online call detail record is within the hotspot duration d, and therefore it is determined to be data within the hotspot.

[0080] If the time segment t is less than the difference between the current time s and the hotspot duration d (i.e., t < (sd)), then this online call detail record is not within the current hotspot duration, and therefore it is identified as data outside the hotspot.

[0081] For example, an online billing system uses Redis to store and process online call detail records (CDRs). Each CDR has a deduplication key, where the first value of the deduplication key is the time shard, indicating the time period in which the CDR was generated. The online billing system also sets a hotspot duration 'd' to define the time period within which data is considered hotspot data.

[0082] There is an online call detail record (CDR) with a time slice of October 27, 2023, at 13:45 in its deduplication key. Since the current time is 14:00, and the hotspot duration is 1 hour, the hotspot period should be from 13:00 to 14:00. Because the time slice 13:45 of the CDR falls within this hotspot period (i.e., 13:45 >= (14:00 - 1:00)), this online CDR is considered data within the hotspot.

[0083] Assuming the current time is 14:00 on October 27, 2023, and the hotspot duration d is 1 hour, there is an online call detail record (CDR) with the time slice in its deduplication key value being 12:30 on October 27, 2023.

[0084] Since the current time is 14:00 and the hotspot duration is 1 hour, the hotspot period is from 13:00 to 14:00. Because the time segment of the call detail record (CDR) at 12:30 is not within this hotspot period (i.e., 12:30 < (14:00 - 1:00)), this online CDR is considered to be outside the hotspot data.

[0085] This application embodiment determines whether a call detail record (CDR) is data within or outside a hotspot based on the relationship between the time segmentation of the online CDR and the current time and the duration of the hotspot, and adopts different billing and deduplication strategies accordingly to optimize the efficiency and accuracy of billing and deduplication.

[0086] In some embodiments, the deduplication service processing mechanism based on the message type corresponding to the online call detail record (CDR) performs billing deduplication on the online CDR, including:

[0087] When the message type of the online call detail record (CDR) is data within a hotspot, determine whether the time offset between the maximum time point of the time slice in the deduplication key value of the online CDR and the current time exceeds the Redis Hash hotspot duration;

[0088] If the time offset does not exceed the Redis Hash hotspot duration, the deduplication key values ​​of the online call detail records are checked using a Bloom filter;

[0089] Based on the deduplication results from the Bloom filter, the online call detail records are deduplicated for billing purposes.

[0090] The duration of Redis Hash hotspots is divided into hours.

[0091] First, check the time offset between the maximum time point of the time slice in the online call detail record (CDR) deduplication key value and the current time. The time offset between the maximum time point of the time slice in the deduplication key value and the current time refers to the difference between the end time (or latest time point) of the time slice recorded in the online CDR deduplication key value and the current system time.

[0092] For example, suppose the time slice in the deduplication key value of an online call detail record (CDR) indicates that the CDR was generated within the first 30 minutes of a certain hour. Then the maximum time point of that time slice is the 30th minute of that hour. If the current time is the 45th minute of that hour, then the time offset is 15 minutes.

[0093] Next, this time offset is compared with the hotspot duration in the Redis Hash table to determine whether the online call detail record (CDR) belongs to the hotspot data. If the time offset is less than or equal to the hotspot duration in the Redis Hash table, it means that the online CDR is within the hotspot range of the Redis Hash table.

[0094] Once an online call detail record (CDR) is determined to be within the hotspot range of the Redis Hash table, the system will first use a Bloom filter for deduplication. A Bloom filter is a highly space-efficient probabilistic data structure that uses a bit array to represent the set and allows for a certain false positive rate. Using a Bloom filter, it's possible to quickly determine whether the deduplication key for an online CDR already exists.

[0095] If the Bloom filter determines that the deduplication key does not exist (i.e., it is not duplicate data), it writes the deduplication key of the online call detail record (CDR) into the Redis Hash table and the Bloom filter. This way, when the online CDR needs to be queried or processed later, the corresponding data can be quickly retrieved from the Redis Hash table and the Bloom filter.

[0096] If the Bloom filter determines that the deduplication key exists (i.e., it may be duplicate data), it will further check whether the deduplication key exists in the Redis Hash table.

[0097] To ensure that online call detail records (CDRs) are not billed twice, Redis hash tables are typically used to store processed CDR information. This information is stored in key-value pairs, where the key is the deduplication key and the value contains the key fields of the CDR associated with that deduplication key, including but not limited to start time, billing ID, session ID, etc.

[0098] Specifically, if the deduplication key exists in the Redis Hash table, the system will search the Redis Hash table to check for associated key-value data. If the Redis Hash table contains key-value data associated with the key field of the deduplication key, the system will further examine the `source_id` field of these key-value data. `source_id` is a field that identifies the source of the call detail record (CDR), such as user ID or device ID.

[0099] If the source_id of the key-value pairs in the duplicate call detail records (CDRs) are the same, it indicates that the existing CDRs originated from the same source. Based on business logic, this could be a duplicate request or a duplicate order. If the source_ids are different, it is considered a non-duplicate order.

[0100] If the key-value pair for the deduplication key is not found in the Redis Hash table, the system determines that it is a non-duplicate order and writes the key-value pair for the deduplication key to the Redis Hash table.

[0101] This application's embodiments utilize Redis Hash tables to efficiently manage online call detail record (CDR) data, achieving accurate billing deduplication. Furthermore, the use of Bloom filters further improves the efficiency and accuracy of deduplication processing.

[0102] In some embodiments, the method further includes:

[0103] If the time offset exceeds the Redis Hash hotspot duration, determine whether the time offset exceeds the Bloom filter hotspot duration.

[0104] If the time offset does not exceed the hotspot duration of the Bloom filter, the online call detail records are deduplicated for billing based on the Bloom filter and the RedisHash table.

[0105] The duration of hotspots in the Bloom filter is divided into days.

[0106] If it is determined that the call detail record (CDR) is not within the Redis Hash table's hotspot range but is within the Bloom filter's hotspot range (i.e., the time offset of the deduplication key is greater than the Redis Hash table's hotspot duration but less than the Bloom filter's hotspot duration), the system will first use the Bloom filter for deduplication.

[0107] If the Bloom filter determines that the deduplication key value is not duplicated, it means that this is a non-duplicate order. This deduplication key value does not exist in the Redis Hash table or the HBase table, so it will be written to the Redis Hash table and the Bloom filter.

[0108] If the Bloom filter determines that the deduplication key value is duplicated, it is necessary to further check the Redis Hash table to see if the deduplication key value actually exists.

[0109] If the Redis Hash table contains key-value data associated with the critical field of the deduplication key, then the source_id field of the deduplication key is used to determine whether it is a duplicate order. If the source_id is the same, it may be a resentment request or a duplicate order; if the source_id is different, it is considered a non-duplicate order.

[0110] If the key-value data associated with the key field of the deduplication key is not found in the Redis Hash table, further deduplication needs to be performed in the HBase table.

[0111] If there is no key-value data associated with the key field of the deduplication key value in the HBase table, it is determined to be a non-duplicate order, and the key-value data of the deduplication key value is only written to the HBase table.

[0112] If the HBase database contains key-value pairs associated with the key fields of this deduplication key, then the source_id field is used to determine whether it is a duplicate order, based on the data in HBase. This will not be elaborated further here.

[0113] In some embodiments, the method further includes:

[0114] If the time offset exceeds the hotspot duration of the Bloom filter, billing deduplication is performed on the online call detail records based on the Redis hash table and the HBase table.

[0115] If the time offset of the deduplication key value is greater than the hot spot duration of the Bloom filter, the Bloom filter will not be accessed again for deduplication.

[0116] Specifically, the system first checks if the deduplication key value exists in the Redis Hash table.

[0117] If the deduplication key value exists in the Redis Hash table, further determination is made based on the data in the Redis Hash table to determine if it is a duplicate order. The logic for determining duplicate orders in the Redis Hash table is the same as in the previous embodiment, which is achieved by identifying the source_id field, and will not be elaborated further here.

[0118] If the deduplication key does not exist in the Redis Hash table, the system will further check if the data exists in HBase. If the deduplication key does not exist in the HBase table, it is determined to be a non-duplicate order, and the key-value data of the deduplication key is written to the HBase table. If the deduplication key exists in the HBase table, it is determined whether it is a duplicate order based on the data in the HBase table, which is also achieved by recognizing the source_id field, and will not be elaborated on here.

[0119] In addition, to optimize performance and memory usage, a data migration thread is added in this embodiment. The Redis Hash table hotspot duration parameter divides the data into hotspot data and non-hotspot data. Hotspot data is stored in a Redis table, and the data migration thread migrates non-hotspot data from the Redis Hash table to the HBase table in near real-time, releasing the memory occupied by Redis.

[0120] refer to Figure 2 , Figure 2 This is the second flowchart illustrating the billing and deduplication method provided by the present invention.

[0121] In this embodiment, after receiving a business deduplication request from the billing application, the deduplication client calls the deduplication service to read and parse the request message. The deduplication service starts corresponding threads to connect to the Redis cluster and the HBSE cluster.

[0122] Specifically, the Redis key for each deduplicated message is calculated. For each key in Redis, the hotspot status is determined based on the key's value. The first value of each key in Redis is the time shard. Based on three parameters—time shard t, hotspot duration d, and current time s—it is determined whether the current key is within the hotspot. If "t >= (sd)", the key belongs to the hotspot; otherwise, it is outside the hotspot. Deduplication is then performed separately for messages within and outside the hotspot according to the corresponding deduplication logic.

[0123] In this embodiment, a three-layer joint deduplication processing method is used to achieve a high-speed joint deduplication service based on the Bloom filter algorithm, the distributed cache Redis, and the large-scale data storage HBase. This provides joint deduplication capabilities for hot and non-hot data, improving the billing accuracy in scenarios where messages are converted to abnormal files and the deduplication accuracy in multi-business scenarios. At the same time, based on the high read and write performance of Bloom filters and Redis, and the support for massive storage by HBase, practical applications have shown that compared with deduplication of Oracle database tables, storage space is saved and deduplication efficiency is improved.

[0124] The billing and de-weighing device provided by the present invention is described below. The billing and de-weighing device described below can be referred to in correspondence with the billing and de-weighing method described above.

[0125] Figure 3 This is a schematic diagram of the billing and de-duplication device provided by the present invention; as shown. Figure 3 As shown, the device includes:

[0126] The first billing and deduplication module 310 is used to respond to a billing and deduplication request and obtain the call detail records to be deduplicated corresponding to the billing and deduplication request.

[0127] The second billing and deduplication module 320 is used to determine that the call detail record to be deduplicated is an offline call detail record, and to perform billing and deduplication on the offline call detail record according to the deduplication key value of the offline call detail record;

[0128] The third billing and deduplication module 330 is used to determine that the call detail record to be deduplicated is an online call detail record, and to determine the message type of the online call detail record based on the deduplication key value of the online call detail record;

[0129] The fourth billing and deduplication module 340 is used to perform billing and deduplication on the online call detail records (CDRs) based on the deduplication service processing mechanism corresponding to the message type of the online CDR.

[0130] The apparatus provided by this invention, in response to a billing and deduplication request, obtains the call detail records (CDRs) to be deduplicated corresponding to the request. If the CDR to be deduplicated is an offline CDR, billing and deduplication are performed on the offline CDR based on its deduplication key value. If the CDR to be deduplicated is an online CDR, the message type of the online CDR is determined based on its deduplication key value. Based on the deduplication service processing mechanism corresponding to the message type of the online CDR, billing and deduplication are performed on the online CDRs containing online messages. This invention, by distinguishing and processing different types of CDRs (offline and online CDRs), ensures a more accurate billing process. For offline CDRs, deduplication and billing based on the deduplication key value avoids duplicate billing; for online CDRs, selecting the appropriate deduplication service processing mechanism based on the message type further ensures billing accuracy.

[0131] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4 As shown, the electronic device may include a processor 410, a communications interface 420, a memory 430, and a communication bus 440, wherein the processor 410, communications interface 420, and memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute a billing deduplication method. This method includes: in response to a billing deduplication request, obtaining the call detail record (CDR) to be deduplicated corresponding to the billing deduplication request; determining that the CDR to be deduplicated is an offline CDR, and performing billing deduplication on the offline CDR according to the deduplication key value of the offline CDR; determining that the CDR to be deduplicated is an online CDR, and determining the message type of the online CDR according to the deduplication key value of the online CDR; and performing billing deduplication on the online CDR based on the deduplication service processing mechanism corresponding to the message type of the online CDR.

[0132] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0133] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the billing and deduplication method provided by the above methods. The method includes: in response to a billing and deduplication request, obtaining the call detail record (CDR) to be deduplicated corresponding to the billing and deduplication request; determining that the CDR to be deduplicated is an offline CDR, and performing billing and deduplication on the offline CDR according to the deduplication key value of the offline CDR; determining that the CDR to be deduplicated is an online CDR, and determining the message type of the online CDR according to the deduplication key value of the online CDR; and performing billing and deduplication on the online CDR based on the deduplication service processing mechanism corresponding to the message type of the online CDR.

[0134] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the billing and deduplication method provided by the above methods. The method includes: in response to a billing and deduplication request, obtaining a call detail record (CDR) to be deduplicated corresponding to the billing and deduplication request; determining that the CDR to be deduplicated is an offline CDR, and performing billing and deduplication on the offline CDR based on the deduplication key value of the offline CDR; determining that the CDR to be deduplicated is an online CDR, and determining the message type of the online CDR based on the deduplication key value of the online CDR; and performing billing and deduplication on the online CDR based on the deduplication service processing mechanism corresponding to the message type of the online CDR.

[0135] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0136] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0137] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A billing deduplication method, characterized in that, include: In response to a billing deduplication request, obtain the call detail records (CDRs) to be deduplicated corresponding to the billing deduplication request; The offline call detail records (CDRs) to be deduplicated are determined to be offline CDRs, and billing deduplication is performed on the offline CDRs based on the deduplication key value of the offline CDRs. The call detail records (CDRs) to be deduplicated are determined to be online CDRs, and the message type of the online CDRs is determined based on the deduplication key value of the online CDRs. Based on the deduplication service processing mechanism corresponding to the message type of the online call detail records (CDRs), billing deduplication is performed on the online CDRs; wherein, for online CDRs with data within a hotspot, a deduplication service processing mechanism using caching or distributed processing is used, and for online CDRs with data outside a hotspot, a deduplication service processing mechanism based on database query is used. The step of determining that the call detail record (CDR) to be deduplicated is an online CDR, and determining the message type of the online CDR based on the deduplication key value of the online CDR, includes: If the time slice in the deduplication key value of the online call detail record is greater than or equal to the difference between the current time and the duration of the hotspot, the message type of the online call detail record is determined to be data within the hotspot. If the time slice in the deduplication key value of the online call detail record (CDR) is less than the difference between the current time and the duration of the hotspot, the message type of the online CDR is determined to be data outside the hotspot.

2. The billing and deduplication method according to claim 1, characterized in that, The step of deduplicating offline call records for billing based on the deduplication key value includes: The deduplication key value of the offline call detail record (CDR) is checked for duplicates against the deduplication key values ​​of all online CDRs. If the deduplication key values ​​of the offline call detail records are not duplicated, the billing deduplication is performed on the offline call detail records based on the deduplication results of the call detail record key values. If the deduplication key value of the offline call detail record (CDR) is duplicated, the offline CDR is marked with a duplicate error code, which indicates that the offline CDR does not need to be billed.

3. The billing and deduplication method according to claim 1, characterized in that, The deduplication service processing mechanism based on the message type corresponding to the online call detail record (CDR) performs billing deduplication on the online CDR, including: When the message type of the online call detail record (CDR) is data within a hotspot, determine whether the time offset between the maximum time point of the time slice in the deduplication key value of the online CDR and the current time exceeds the Redis Hash hotspot duration; If the time offset does not exceed the Redis Hash hotspot duration, the deduplication key values ​​of the online call detail records are checked using a Bloom filter; Based on the deduplication results from the Bloom filter, the online call detail records are deduplicated for billing purposes.

4. The billing and deduplication method according to claim 3, characterized in that, The method further includes: If the time offset exceeds the Redis Hash hotspot duration, determine whether the time offset exceeds the Bloom filter hotspot duration. If the time offset does not exceed the hotspot duration of the Bloom filter, the online call detail records are deduplicated for billing based on the Bloom filter and the Redis Hash table.

5. The billing and deduplication method according to claim 4, characterized in that, The method further includes: If the time offset exceeds the hotspot duration of the Bloom filter, billing deduplication is performed on the online call detail records based on the Redis hash table and the HBase table.

6. A billing and de-weighing device, characterized in that, include: The first billing and deduplication module is used to respond to a billing and deduplication request and obtain the call detail records to be deduplicated corresponding to the billing and deduplication request. The second billing and deduplication module is used to determine that the call detail record to be deduplicated is an offline call detail record, and to perform billing and deduplication on the offline call detail record according to the deduplication key value of the offline call detail record; The third billing and deduplication module is used to determine that the call detail record (CDR) to be deduplicated is an online CDR, and to determine the message type of the online CDR based on the deduplication key value of the online CDR. The fourth billing and deduplication module is used to perform billing and deduplication on the online call detail records (CDRs) based on the deduplication service processing mechanism corresponding to the message type of the online CDR. Specifically, for online call detail records (CDRs) within hotspots, a deduplication service mechanism using caching or distributed processing is employed; for online CDRs outside hotspots, a deduplication service mechanism based on database queries is used. The step of determining that the call detail record (CDR) to be deduplicated is an online CDR, and determining the message type of the online CDR based on the deduplication key value of the online CDR, includes: If the time slice in the deduplication key value of the online call detail record is greater than or equal to the difference between the current time and the duration of the hotspot, the message type of the online call detail record is determined to be data within the hotspot. If the time slice in the deduplication key value of the online call detail record (CDR) is less than the difference between the current time and the duration of the hotspot, the message type of the online CDR is determined to be data outside the hotspot.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the billing and deduplication method as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the billing and deduplication method as described in any one of claims 1 to 5.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the billing and deduplication method as described in any one of claims 1 to 5.

Images