Pricing test method, device and equipment for a billing system and medium

Abstract

The disclosure provides a pricing test method, device, equipment and medium for a charging system, and belongs to the technical field of function test. The method comprises the following steps: generating verification test sample data, wherein the verification test sample data comprises verification test user data, verification test call data and result verification data; determining whether a first condition is met; providing the verification test user data and the verification test call data to the charging system; obtaining execution result data of the charging system in performing pricing test on the verification test call data according to the verification test user data; and determining a pricing test result of the charging system according to the result verification data and the execution result data. The disclosure can effectively improve the verification accuracy of the verification result of the pricing correctness, and effectively reduce the verification time and the verification workload.

Description

Technical Field

[0001] This disclosure relates to the field of functional testing technology, and in particular to a batch testing method, apparatus, equipment and medium for billing systems. Background Technology

[0002] In related technologies, billing systems typically verify the correctness of pricing through manual testing. This mainly includes two testing methods and one result verification method. Testing Method 1: Based on a test number, call detail records (CDRs) are generated by making calls, and the billing system processes the CDRs for pricing. Testing Method 2: Using a test number, a test CDR file is generated using a CDR generation tool, and the billing system processes the CDRs for pricing. After the billing system completes the pricing, the result is saved to the database. Result Verification Method: Maintenance personnel verify the pricing result using a result verification tool to confirm its correctness.

[0003] In this approach, the accuracy of the verification results for the correctness of the price is low, the verification time is long, and the workload required for verification is large. Summary of the Invention

[0004] This disclosure aims to at least partially address one of the technical problems in the related art.

[0005] Therefore, the purpose of this disclosure is to provide a batch pricing test method, apparatus, computer equipment and storage medium for a billing system, which can effectively improve the verification accuracy of the batch pricing correctness verification results and effectively reduce the verification time and the workload required for verification.

[0006] To achieve the above objectives, the first aspect of this disclosure provides a pricing test method for a billing system, comprising:

[0007] Generate verification test sample data, wherein the verification test sample data includes: verification test user information, verification test call detail records and result verification data;

[0008] Once the first condition is met, the verification test user information and the verification test call detail record are provided to the billing system.

[0009] Obtain the execution result data of the billing system performing batch pricing tests on the verification test call details based on the verification test user data; and

[0010] Based on the verification data and the execution result data, the batch pricing test results of the billing system are determined.

[0011] To achieve the above objectives, the second aspect of this disclosure provides a billing system batch testing apparatus, comprising:

[0012] The generation module is used to generate verification test sample data, wherein the verification test sample data includes: verification test user information, verification test call detail records and result verification data;

[0013] The first determining module is used to determine that the first condition is met, and to provide the verification test user information and the verification test call detail record to the billing system.

[0014] The acquisition module is used to acquire the execution result data of the billing system's batch pricing test on the verification test call detail records based on the verification test user information; and

[0015] The second determining module is used to determine the batch pricing test results of the billing system based on the result verification data and the execution result data.

[0016] The computer device proposed in the third aspect of this disclosure includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the batch pricing test method for a billing system as proposed in the first aspect of this disclosure.

[0017] The fourth aspect of this disclosure provides a non-transitory computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the batch testing method for a billing system as proposed in the first aspect of this disclosure.

[0018] The fifth aspect of this disclosure provides a computer program product that, when executed by a processor, performs a batch pricing test method for a billing system as described in the first aspect of this disclosure.

[0019] The pricing test method, apparatus, computer equipment, and storage medium for a billing system disclosed herein generate verification test sample data, which includes verification test user information, verification test call detail records (CDRs), and result verification data. After determining that a first condition is met, the verification test user information and CDRs are provided to the billing system. The execution result data of the billing system's pricing test on the CDRs based on the verification test user information is obtained. Based on the result verification data and the execution result data, the pricing test result of the billing system is determined. This effectively improves the accuracy of the pricing correctness verification results and effectively reduces verification time and workload.

[0020] Additional aspects and advantages of this disclosure will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this disclosure. Attached Figure Description

[0021] The above and / or additional aspects and advantages of this disclosure will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, in which:

[0022] Figure 1 This is a flowchart illustrating a pricing test method for a billing system according to an embodiment of this disclosure;

[0023] Figure 2 This is a flowchart illustrating a batch pricing test method for a billing system according to another embodiment of this disclosure;

[0024] Figure 3 This is a flowchart illustrating the verification test sample data cleaning process proposed in this disclosure;

[0025] Figure 4 This is a flowchart illustrating the automated verification process for verification tests as presented in this disclosure;

[0026] Figure 5 This is a schematic diagram of the automated execution process for verification testing based on this disclosure;

[0027] Figure 6 This is a flowchart illustrating a batch pricing test method for a billing system according to another embodiment of this disclosure;

[0028] Figure 7 This is a schematic diagram of the process for generating verification test sample data based on the present disclosure;

[0029] Figure 8 This is a flowchart illustrating a batch pricing test method for a billing system according to another embodiment of this disclosure;

[0030] Figure 9 This is a flowchart illustrating the verification test sample data processing based on the present disclosure;

[0031] Figure 10 This is a schematic diagram of the processing flow of abnormal description information proposed in this disclosure;

[0032] Figure 11 This is a schematic diagram of the verification and testing process for the billing system proposed in this disclosure;

[0033] Figure 12 This is a structural diagram of the automated verification system for the billing production system proposed in this disclosure;

[0034] Figure 13 This is a flowchart illustrating the automated verification process proposed in this disclosure;

[0035] Figure 14 This is a schematic diagram of the structure of a pricing test device for a billing system according to an embodiment of this disclosure;

[0036] Figure 15 A block diagram of an exemplary computer device suitable for implementing embodiments of the present disclosure is shown. Detailed Implementation

[0037] Embodiments of this disclosure are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are used only to explain this disclosure, and should not be construed as limiting this disclosure. Rather, embodiments of this disclosure include all variations, modifications, and equivalents falling within the spirit and scope of the appended claims.

[0038] Figure 1 This is a flowchart illustrating a pricing test method for a billing system proposed in one embodiment of this disclosure.

[0039] It should be noted that the execution subject of the billing system batch testing method in this embodiment is the billing system batch testing device. This device can be implemented by software and / or hardware. The device can be configured in a computer device, which may include, but is not limited to, a terminal, a server, etc. For example, the terminal may be a mobile phone, a PDA, etc.

[0040] like Figure 1 As shown, the batch pricing test method for the billing system includes:

[0041] S101: Generate verification test sample data, which includes: verification test user information, verification test call detail records and result verification data.

[0042] Among them, the verification test sample data refers to the sample data used for batch testing of the billing system.

[0043] Verification test user data refers to the user information records created for the test numbers participating in the pricing test. For example, this data includes, but is not limited to, the following: user identifier, user address information, number plan type, plan start and end dates, etc., without restriction.

[0044] Verification test call detail records (CDRs) refer to billing records generated by simulating the communication behavior of test users. They can contain detailed information such as various operations performed by the test user during communication, call duration, number of SMS messages, and data traffic. Verification test CDRs can simulate the communication behavior of real users so that the testing billing system can correctly generate the corresponding billing records.

[0045] Verification data refers to data used to verify the billing process by comparing the billing results generated by the billing system with the expected results. It includes information such as the expected billing amount, cost allocation rules, and billing details. Verification data can be used to verify whether the billing system processes test call details as expected and generates accurate billing results.

[0046] In this embodiment of the disclosure, a communication link between the execution subject of this embodiment and the big data server can be established in advance, and then the verification test sample data can be obtained from the big data server. Alternatively, the verification test sample data can be obtained based on a third-party sample data generation device, and there are no restrictions on this.

[0047] In this embodiment of the disclosure, when generating verification test sample data, reliable data support can be provided for the batch pricing test process used in the billing system.

[0048] S102: Once the first condition is met, provide the billing system with the verification test user information and verification test call detail records.

[0049] The first condition refers to the triggering conditions pre-configured for the verification test of the billing system.

[0050] For example, in this embodiment of the disclosure, multiple verification test time points can be pre-configured. When the running time of the billing system reaches the verification test time point, it is determined that the first condition is met. Alternatively, the first condition can be determined based on a third-party device, without any limitation.

[0051] The billing system refers to a system used to record information related to user call charges. This disclosure does not limit the type of billing system.

[0052] In other words, the embodiments of this disclosure can provide the billing system with verification test user information and verification test call details by using a first condition as a trigger during the batch pricing test of the billing system, thereby providing timely data support for the billing system to conduct batch pricing tests.

[0053] S103: Obtain the execution result data of the billing system's batch pricing test on the verification test call detail records based on the verification test user data.

[0054] Among them, the execution result data refers to the data obtained by the billing system from the batch pricing test of the verification test call detail records based on the verification test user data.

[0055] Understandably, in order to ensure the reliability, accuracy and stability of the billing system, it is usually necessary to frequently conduct batch pricing tests on the billing system. When the execution result data of the billing system performing batch pricing tests on the verification test call details based on the verification test user data is obtained, the obtained execution result data can provide reliable reference information for evaluating the performance of the billing system.

[0056] S104: Determine the batch pricing test results of the billing system based on the result verification data and execution result data.

[0057] Among them, the batch pricing test results refer to the results obtained by conducting batch pricing tests on the billing system, which can be used to evaluate the reliability of the billing system.

[0058] In this embodiment of the disclosure, when determining the batch test result of the billing system based on the result verification data and the execution result data, the result verification data and the execution result data may be input into a pre-trained machine learning model to obtain the batch test result of the billing system. Alternatively, the result verification data and the execution result data may be processed based on a combination of numerical and graphical methods to obtain the batch test result of the billing system. Or, the batch test result of the billing system may be determined based on any other possible method, without any limitation.

[0059] Understandably, billing systems typically rely on a complex set of billing rules to determine charges. The methods disclosed herein can verify whether these rules function as intended and whether charges are calculated correctly. These methods can also promptly identify errors and vulnerabilities in the billing system, preventing overcharging or undercharging that could cause losses to customers or negatively impact company finances. Since billing systems involve significant data processing and interaction, the methods disclosed herein can verify whether the system correctly receives, stores, and processes data to ensure the accuracy of billing information. Furthermore, the performance and capacity of the billing system can be evaluated, identifying its performance under high loads and determining its ability to handle future business growth. Because billing systems directly affect customer billing and payments, the methods disclosed herein can maintain system stability, improve user satisfaction, and prevent unnecessary disputes and complaints.

[0060] In this embodiment, by generating verification test sample data, which includes verification test user information, verification test call detail records (CDRs), and result verification data, and determining that the first condition is met, the verification test user information and verification test CDRs are provided to the billing system. The execution result data of the billing system's batch pricing test on the verification test CDRs based on the verification test user information is obtained. Based on the result verification data and the execution result data, the batch pricing test result of the billing system is determined. This effectively improves the verification accuracy of the batch pricing correctness verification result and effectively reduces the verification time and workload required for verification.

[0061] Figure 2 This is a flowchart illustrating a pricing test method for a billing system proposed in another embodiment of this disclosure.

[0062] like Figure 2 As shown, the batch pricing test method for the billing system includes:

[0063] S201: Delete the verification test user information associated with the virtual number information used in the previous test from the business database.

[0064] The business database refers to the database that stores and manages data related to the billing process. For example, it can be used to store customer information, service usage data, cost calculation rules, payment records, and other data.

[0065] The virtual number information refers to the number information of virtual users pre-configured for the billing system's batch testing process. This virtual number information may, for example, include mobile phone number ranges that are not used by real users.

[0066] In other words, before conducting the current round of price testing, the present disclosure embodiment can delete the verification test user data associated with the virtual number information used in the previous test from the business database, thereby avoiding excessive verification test user data obtained from multiple rounds of testing, which would affect the storage capacity of the business database.

[0067] S202: Delete the result verification data associated with the virtual number information used in the previous test from the verification test database.

[0068] The verification test database refers to a pre-configured database used to store verification test-related data.

[0069] In other words, before conducting the current round of price testing, this embodiment of the present disclosure can delete the result verification data associated with the virtual number information used in the previous test from the verification test database, thereby affecting the result verification data associated with the virtual number information used in the previous test in the current price testing process.

[0070] S203: Delete the execution result data associated with the virtual number information used in the previous test from the billing system's database.

[0071] In other words, before conducting the current round of pricing tests, this embodiment of the present disclosure can delete the execution result data associated with the virtual number information used in the previous test from the billing system's database, thereby effectively improving the usability of the data stored in the billing system's database.

[0072] S204: Delete the verification test user information, verification test call detail records, and result verification data associated with the virtual number information used in the previous test from local storage.

[0073] In other words, before conducting this round of price approval testing, the present disclosure embodiment can delete the verification test user data, verification test call detail records, and result verification data associated with the virtual number information used in the previous test from the local storage. This allows for timely cleanup of the verification test user data, verification test call detail records, and result verification data associated with the virtual number information used in the previous test stored in the local storage, thereby avoiding data accumulation that could affect the efficiency of this price approval test.

[0074] For example, such as Figure 3 As shown, Figure 3 This is a flowchart illustrating the verification test sample data cleaning process proposed in this disclosure, which includes the following steps:

[0075] (1) Delete the verification test user information from the business database based on the verification test number in the virtual number information;

[0076] (2) Delete the verification test result data and the billing system pricing result data from the verification test database and the billing business system database based on the verification test number in the virtual number information;

[0077] (3) Delete the verification test user information file, the verification test user call detail record file, and the verification test result data file in the local directory.

[0078] In other words, in this embodiment of the present disclosure, before generating verification test sample data based on virtual number information and verification test data template, the verification test user information associated with the virtual number information used in the previous test can be deleted from the business database, the result verification data associated with the virtual number information used in the previous test can be deleted from the verification test database, the execution result data associated with the virtual number information used in the previous test can be deleted from the billing system database, and the verification test user information, verification test call detail records, and result verification data associated with the virtual number information used in the previous test can be deleted from the local storage. As a result, data related to the historical test process in multiple data storage units can be cleaned up in a timely manner, so as to effectively avoid data accumulation in the database and affect the current batch pricing test process.

[0079] S205: Generate verification test sample data based on the virtual number information and the verification test data template. The virtual number information and the real number information are different. The verification test data template includes at least one of the following: user information data template; verification call detail record data template; result verification data template.

[0080] Among them, the verification test data template can refer to a pre-configured template used to generate verification test sample data.

[0081] Among them, real number information refers to the number information that a real user has already used.

[0082] In this embodiment, the virtual number information and the real number information are different, which can effectively avoid the overlap between the virtual number information and the real number information, thus affecting the user experience of the real user corresponding to the real number information.

[0083] The user profile data template refers to a pre-configured data template used to generate user profiles for verification testing.

[0084] Among them, the verification call detail record (CDR) data template refers to the pre-configured data template used to generate verification test CDRs.

[0085] Among them, the result verification data template refers to the pre-configured data template used to generate result verification data.

[0086] In this embodiment of the disclosure, when generating verification test sample data based on virtual number information and verification test data template, it can be based on a pre-written code script, using functions such as string processing and random number generation of programming languages ​​to generate verification test sample data based on virtual number information and verification test data template. Alternatively, it can be based on a template engine library to generate verification test sample data based on virtual number information and verification test data template. There are no restrictions on this.

[0087] Optionally, in some embodiments, the virtual number information includes: a virtual number segment, and / or a virtual number user identifier, and / or a virtual International Mobile Subscriber Identity (IMSI). This can significantly improve the practicality of virtual number information in the billing system's batch testing process.

[0088] Virtual number segments refer to fictitious number segments that are not used by real users.

[0089] The virtual number user identifier (USER_ID) is a unique identifier assigned to a corresponding virtual user in the billing system or other business systems. It is used to identify and manage the user's account and related information within the system. USER_ID is typically generated automatically by the system or according to certain rules, and can be used for business operations such as user identification, billing management, and operational analysis.

[0090] The International Mobile Subscriber Identity (IMSI) is a unique identifier used to authenticate virtual users and connect them to mobile networks.

[0091] In other words, in this embodiment of the present disclosure, verification test sample data can be generated based on virtual number information and verification test data template. The virtual number information and the real number information are different. The verification test data template includes at least one of the following: user information data template; verification call detail record data template; and result verification data template. Therefore, the standardization of the obtained verification test sample data can be effectively improved based on the verification test data template, thereby improving the reliability of the verification test sample data in the sample testing process.

[0092] S206: Once the first condition is met, provide the billing system with the verification test user information and verification test call detail records.

[0093] Optionally, in some embodiments, when it is determined that the first condition is met, it may be determined that the duration after the last execution of the verification test of the billing system has reached the second time interval. Thus, the timed verification test of the billing system can be carried out based on the second time interval, which can effectively improve the automation of the verification test process.

[0094] The second timing duration can refer to the pre-configured interval between the previous execution of the billing system verification test and the current execution of the billing system verification test. For example, it can be one day or one hour, and there is no restriction on this.

[0095] Optionally, in some embodiments, when the first condition is determined to be met, an execution instruction may be received. The execution instruction is used to trigger the verification test of the billing system. Thus, the verification test of the billing system can be triggered in a timely manner based on the execution instruction, thereby effectively improving the robustness of the verification test process.

[0096] S207: Obtain the execution result data of the billing system's batch pricing test on the verification test call detail records based on the verification test user data.

[0097] For a detailed description of S207, please refer to the above embodiments, which will not be repeated here.

[0098] S208: Link and store virtual number information and verification test user data in the business database.

[0099] S209: Link and store the virtual number information and result verification data in the verification test database.

[0100] S210: Link and store the virtual number information and execution result data in the billing system's database.

[0101] S211: Associate and store the virtual number information, verification test user information, verification test call detail records, and result verification data in local storage.

[0102] For example, automated verification testing compares the data generated by the billing system based on user information and call detail records with the verification results according to comparison rules to determine whether there are any anomalies in the billing system. Figure 4 As shown, Figure 4 This is a flowchart illustrating the automated verification process for verification tests as presented in this disclosure, which includes the following steps:

[0103] (1) Load the approved user call detail records (CDRs) into the verification test database;

[0104] (2) Load the result verification data into the verification test database;

[0105] (3) Compare the batch pricing result data (i.e. the execution result data mentioned above) in the billing system business database with the result verification data according to the comparison rules, and load the result data that match and the result data that do not match into the verification database.

[0106] In other words, in this embodiment of the present disclosure, after obtaining the virtual number information, verification test user information, result verification data, execution result data, and verification test call detail records, the virtual number information and verification test user information can be associated and stored in the business database, the virtual number information and result verification data can be associated and stored in the verification test database, the virtual number information and execution result data can be associated and stored in the billing system database, and the virtual number information, verification test user information, verification test call detail records, and result verification data can be associated and stored in local storage. Thus, the relevant data obtained from the batch pricing test can be stored in a timely manner so that the relevant data can be quickly retrieved during the batch pricing test.

[0107] S212: Determine the batch pricing test results of the billing system based on the result verification data and execution result data.

[0108] Optionally, in some embodiments, the execution result data and the result verification data each include data in at least one of the following dimensions: cumulative data, balance data, deduplication data, list data, reminder data, credit control data, speed limit data, and deduction data. This can greatly enrich the indication content of the result verification data and the execution result data, thereby effectively improving the comprehensiveness of the batch pricing test process and the reliability of the obtained batch pricing test results.

[0109] Cumulative data can refer to cumulative values ​​related to user communication services, such as voice call duration, number of SMS messages, and data traffic consumption.

[0110] The balance data can refer to the remaining available amount of money or resources in a user's account or package.

[0111] Among them, the deduplication data can be used to check and prevent duplicate billing, preventing duplicate billing caused by system errors or user operation problems.

[0112] The list data can refer to detailed information about a user's communication activities, such as call records, SMS sending records, and data usage records.

[0113] Among them, the reminder data is used to send bill reminders and notifications to users, such as bill due reminders and insufficient balance reminders.

[0114] Among them, information control data can be data used to control user communication services, such as determining whether a user has communication permissions or package restrictions.

[0115] Among them, speed limit data can be data that restricts the user's communication speed, reducing the communication speed according to the user's payment status or package rules.

[0116] Among them, the deduction data can be the cost data calculated and deducted based on the user's communication behavior and billing policy.

[0117] For example, such as Figure 5 As shown, Figure 5 Based on the verification test automation execution flowchart proposed in this disclosure, the verification test automation execution is used to periodically trigger or manually trigger the execution of the verification test automation tool through the interface in real time, and includes the following steps:

[0118] (1) Use crontab expressions to set the time for periodically executing the verification test tool (i.e., the duration after the last execution of the billing system verification test is determined to reach the second time interval), and provide an interface to support manual real-time triggering of the verification test execution through the interface (i.e., receiving the execution command).

[0119] (2) Load the verification test user information and verification test result data into the database;

[0120] (3) The billing system reads the verification test call detail records and performs call detail record pricing based on the verification test user information;

[0121] (4) Filter the pricing results of the verification test call records to prevent the downstream accounting system from processing the approved verification test call records. Generate a result verification data file from the pricing results of these verification test call records, and filter out the approved verification test call records.

[0122] In this embodiment, by deleting verification test user data associated with the virtual number information used in the previous test from the business database, deleting result verification data associated with the virtual number information used in the previous test from the verification test database, deleting execution result data associated with the virtual number information used in the previous test from the billing system database, and deleting verification test user data, verification test call detail records, and result verification data associated with the virtual number information used in the previous test from local storage, data related to the historical test process can be cleaned up in multiple data storage units in a timely manner, effectively avoiding data accumulation in the database that could affect the current batch pricing test process. Verification test sample data is generated based on the virtual number information and the real number information. The virtual number information and the real number information are different. The verification test data template includes at least one of the following: user data template; verification call detail record data template; and result verification data template. Therefore, the standardization of the obtained verification test sample data can be effectively improved based on the verification test data template, thereby improving the reliability of the verification test sample data in the sample testing process. The first condition can be met if the duration since the last execution of the billing system verification test reaches the second time interval. This allows for timed verification testing of the billing system based on the second time interval, effectively improving the automation of the verification testing process. Alternatively, the first condition can be met by receiving an execution command, which triggers the execution of the billing system verification test. This allows for timely triggering of the verification test, effectively improving the robustness of the verification testing process. Virtual number information includes: virtual number segments, and / or virtual number user identifiers, and / or virtual International Mobile Subscriber Identity (IMSI). This significantly enhances the practicality of virtual number information in the billing system's batch testing process. By associating virtual number information and verification test user data in the business database, virtual number information and result verification data in the verification test database, virtual number information and execution result data in the billing system database, and virtual number information, verification test user data, verification test call detail records, and result verification data in local storage, relevant data obtained from the pricing test can be stored in a timely manner for quick retrieval during the pricing test process. Execution result data and result verification data each include at least one of the following dimensions: cumulative data, balance data, duplicate data, list data, reminder data, credit control data, rate limiting data, and deduction data. This significantly enriches the indicative content of the result verification data and execution result data, effectively improving the comprehensiveness of the pricing test process and enhancing the reliability of the obtained pricing test results.

[0123] Figure 6This is a flowchart illustrating a pricing test method for a billing system proposed in another embodiment of this disclosure.

[0124] like Figure 6 As shown, the batch pricing test method for the billing system includes:

[0125] S601: Generate verification test user information based on virtual number information and user data template.

[0126] In other words, in the embodiments of this disclosure, when generating verification test user data, the user data template can be filled with virtual number information to obtain verification test user data.

[0127] Optionally, in some embodiments, when generating verification test user data based on virtual number information and user data template, the virtual number information can be filled into the user data template, and one or more first variables to be filled in the user data template can be filled to obtain the user data template to be instantiated. The user data template to be instantiated is then instantiated to obtain the verification test user data. This can effectively improve the efficiency of obtaining verification test user data and ensure the practicality of the obtained verification test user data.

[0128] The first variable to be filled refers to the variable pre-configured in the user profile data template. The specific value of this first variable to be filled can be determined based on the virtual number information in the application scenario.

[0129] Among them, the user data template to be instantiated refers to the data template obtained by filling in the user data template with virtual number information.

[0130] In this embodiment of the disclosure, the user data template to be instantiated is instantiated by combining the general structure and format in the template with the actual data to generate verification test user data that meets the requirements of batch testing.

[0131] In this embodiment of the disclosure, the instantiation process can be customized according to specific needs and application scenarios, such as randomly selecting data, generating data according to rules, or obtaining data from real data sources, etc., without any restrictions.

[0132] S602: Generate a verification test call detail record (CDR) based on the virtual number information and the verification CDR data template.

[0133] In other words, in the embodiments of this disclosure, when generating verification test call detail records, the verification test call detail records can be obtained by filling the verification call detail record data template based on virtual number information.

[0134] Optionally, in some embodiments, when generating a verification test call detail record (CDR) based on virtual number information and a verification CDR data template, the virtual number information can be filled into the verification CDR data template, and one or more second variables to be filled into the verification CDR data template can be filled to obtain a verification CDR data template to be instantiated. The verification CDR data template to be instantiated is then instantiated to obtain the verification test CDR. Thus, verification test CDRs can be quickly generated based on the verification CDR data template, which can effectively improve the data standardization of the obtained verification test CDRs.

[0135] The second variable to be filled refers to the variable pre-configured in the verification call detail record (CDR) data template. The specific value of this second variable can be determined based on the virtual number information in the application scenario.

[0136] The call detail record (CDR) data template to be instantiated and verified refers to the data template obtained after filling the verification CDR data template with virtual number information.

[0137] S603: Generate result verification data based on virtual number information and result verification data template.

[0138] In other words, in this embodiment of the present disclosure, when generating result verification data, the result verification data can be generated by filling the result verification data template according to the virtual number information.

[0139] Optionally, in some embodiments, when generating result verification data based on virtual number information and result verification data template, the virtual number information can be filled into the result verification data template, and one or more third variables to be filled into the result verification data template can be filled to obtain the result verification data template to be instantiated. The result verification data template to be instantiated is then instantiated to obtain the result verification data. Thus, the data standardization of the obtained result verification data can be guaranteed based on the result verification data, which can effectively improve the applicability of the result verification data in the batch pricing test process.

[0140] In other words, in this embodiment of the present disclosure, verification test user data can be generated based on virtual number information and user data template, verification test call detail records can be generated based on virtual number information and verification call detail record data template, and result verification data can be generated based on virtual number information and result verification data template. Thus, the corresponding verification test data templates can be filled based on virtual number information to quickly and accurately generate corresponding types of verification test sample data.

[0141] For example, such as Figure 7 As shown, Figure 7The flowchart for generating verification test sample data according to this disclosure is as follows: The verification test sample data generation is used to generate verification test user information, verification test user call detail records, result verification data, etc., based on the verification test data template and verification test number range, including the following steps:

[0142] (1) Pre-made verification test data templates, including user information templates, user call detail records templates, and result verification data templates;

[0143] (2) Prepare a verification test mobile phone number segment that does not exist on the current network (number segment starting with "2") to ensure that the verification test mobile phone number does not exist on the current network and to ensure the security of the verification test;

[0144] (3) Render the test data template based on the verification test mobile phone number, populate the verification test data template variables, instantiate the rendered verification test data template and generate verification test user information, verification test user call detail record data and result verification data, and store them in the specified location.

[0145] S604: Once the first condition is met, provide the billing system with the verification test user information and verification test call detail records.

[0146] S605: Obtain the execution result data of the billing system's batch pricing test on the verification test call detail records based on the verification test user data.

[0147] S606: Determine the batch pricing test results of the billing system based on the result verification data and execution result data.

[0148] For a detailed description of S604-S606, please refer to the above embodiments, which will not be repeated here.

[0149] In this embodiment, verification test user data is generated based on virtual number information and user data templates. Verification test call details (CDRs) are generated based on virtual number information and verification call detail record (CDR) data templates. Result verification data is generated based on virtual number information and result verification data templates. Thus, corresponding verification test data templates can be filled based on virtual number information to quickly and accurately generate corresponding types of verification test sample data. By filling virtual number information into the user data template and filling one or more first variables to be filled in the user data template, a user data template to be instantiated is obtained. Instantiation processing of the user data template to be instantiated yields the verification test user data. This effectively improves the efficiency of obtaining verification test user data and ensures the practicality of the obtained verification test user data. By filling virtual number information into the verification CDR data template and filling one or more second variables to be filled in the verification CDR data template, a verification CDR data template to be instantiated is obtained. Instantiation processing of the verification CDR data template to be instantiated yields the verification test CDR. Thus, verification test CDRs can be quickly generated based on the verification CDR data templates, effectively improving the data standardization of the obtained verification test CDRs. By filling virtual number information into the result verification data template and filling one or more third variables to be filled in the result verification data template, a result verification data template to be instantiated is obtained. The result verification data template to be instantiated is then instantiated to obtain result verification data. Thus, the data standardization of the obtained result verification data can be guaranteed based on the result verification data, which can effectively improve the applicability of the result verification data in the batch pricing test process.

[0150] Figure 8 This is a flowchart illustrating a pricing test method for a billing system proposed in another embodiment of this disclosure.

[0151] like Figure 8 As shown, the batch pricing test method for the billing system includes:

[0152] S801: Generate verification test sample data, which includes: verification test user information, verification test call detail records and result verification data.

[0153] For a detailed description of S801, please refer to the above embodiments, which will not be repeated here.

[0154] S802: Determine that the first condition is met, add a preset mark to the verification test call detail record (CDR). The preset mark is used to instruct the billing system to perform batch pricing tests on the CDR and to instruct that no billing consolidation processing is performed on the CDR.

[0155] Among them, billing consolidation refers to the process of combining the costs from multiple dialog boxes and generating a total cost.

[0156] Understandably, in practical applications, billing systems may process a large number of call records (CRTs) and perform billing consolidation to generate overall cost information. However, during the batch pricing testing phase, it is usually only necessary to verify the billing results of a single CRT, without needing to involve the combined calculation of multiple CRTs. Therefore, in this embodiment of the disclosure, when the first condition is determined to be met, a preset flag can be added to the verification test CRT to instruct the billing system to perform batch pricing testing on the verification test CRT and to instruct that billing consolidation should not be performed on the verification test CRT, thereby simplifying the testing process and focusing on testing the batch pricing function of the billing system.

[0157] S803: Provides the billing system with verification test user information and verification test call detail records with preset tags.

[0158] For example, to avoid incurring charges and resulting arrears due to automated verification of call detail records in the production system, the verification test sample data can be processed, such as... Figure 9 As shown, Figure 9 The flowchart of the verification test sample data processing proposed in this disclosure can generate pre-processed call detail records (CDRs) with the "AutoCheck" identifier (i.e. the aforementioned preset identifier) ​​added to the CDR name. CDRs with this identifier are directly entered into the result table through temporary storage after the CDR is processed, without being transmitted to the accounting host. That is, only the CDR is processed without accounting.

[0159] Mobile phone number: When executing on the production system, the first three digits of the mobile phone number are replaced with 222, for example: 22250134634.

[0160] user_id: In production, user_id is replaced with 59X+9999999, for example: 591999999900001.

[0161] In production, IMSI:imsi is executed by replacing the first four digits with 2222.

[0162] The accumulated data is routed to different production databases based on the modulo of user_id for initialization.

[0163] It is understandable that when generating test cases for automated verification tools, it is necessary to initialize the test users' information and accumulated data. This involves deleting and modifying user information and accumulated data, which poses a risk. Therefore, a special number segment needs to be allocated specifically for the use of automated verification tools.

[0164] After the test users and test call details are processed through the normal pricing process, the pricing results are stored in the user cumulative volume database. The automated verification tool compares the cumulative volume results with the target cumulative volume data generated by the test cases to confirm whether there are any problems with the pricing application.

[0165] In the error order recovery process, you can choose to discard the order directly, not retry (normally retry 3 times), or not put it into the warehouse (put it into the warehouse after 3 failed retries).

[0166] In terms of process assurance, automated test call detail records (CDRs) can be filtered when counting the number of CDRs.

[0167] In other words, in this embodiment of the present disclosure, after determining that the first condition is met, a preset mark can be added to the verification test call detail record (CDR). The preset mark is used to instruct the billing system to perform batch pricing tests on the verification test CDR and to instruct that the verification test CDR should not be consolidated. The billing system is provided with the verification test user information and the verification test CDR with the preset mark added. Thus, the billing system can be instructed to perform batch pricing tests on the verification test CDR based on the preset mark, without consolidating the verification test CDR, thereby effectively improving the rationality of the batch pricing test process and effectively avoiding the waste of computing resources.

[0168] S804: Obtain the execution result data of the billing system's batch pricing test on the verification test call detail records based on the verification test user data.

[0169] For a detailed description of S804, please refer to the above embodiments, which will not be repeated here.

[0170] S805: Determine whether the comparison rule indicators are met between the execution result data and the result verification data.

[0171] The comparison rule metrics can refer to metrics pre-configured based on comparison rules between execution result data and result verification data. In this embodiment, verification test result comparison rules can be written in Python to improve verification execution efficiency.

[0172] For example, a comparison rule metric could refer to the similarity value between the execution result data and the result verification data.

[0173] S806: If the execution result data and the result verification data meet the comparison rule indicators, then the batch pricing test result indicates that the billing system meets the test indicators.

[0174] Among them, the test indicators can refer to the indicators that the billing system meets the preset batch pricing test requirements.

[0175] In other words, in this embodiment of the disclosure, when the comparison rule indicators are met between the execution result data and the result verification data, the batch pricing test result can be determined to indicate that the billing system meets the test indicators.

[0176] S807: If the execution result data and the result verification data do not meet the comparison rule indicators, then the batch pricing test result indicates that the billing system does not meet the test indicators.

[0177] In this embodiment of the disclosure, after determining that the batch pricing test results indicate that the billing system does not meet the test indicators, for example, the current billing system can be shut down and a backup billing system can be enabled at the same time, thereby effectively avoiding billing system errors that affect user experience.

[0178] Optionally, in some embodiments, after determining that the batch pricing test results indicate that the billing system does not meet the test indicators, anomaly description information can be generated based on the execution result data and result verification data. The anomaly description information is used to describe the situation in which the billing system generates anomalies and to provide anomaly description information. Thus, anomaly description information can be generated and provided in a timely manner after determining that the batch pricing test results indicate that the billing system does not meet the test indicators, so as to quickly locate the anomaly point and perform fault analysis and repair.

[0179] Optionally, in some embodiments, when generating anomaly description information based on execution result data and result verification data, it may be necessary to determine the duration after the last determination of the billing system's batch pricing test result reaches a first time interval, determine the result difference data between the execution result data and the result verification data, and generate anomaly description information based on the result difference data. This enables periodic scanning of the result difference data, which can effectively improve the robustness of the billing system.

[0180] The first timing duration can refer to the pre-configured period of the result difference data between the execution result data and the result verification data.

[0181] Among them, result difference data can refer to data that describes the differences between execution result data and result verification data.

[0182] Optionally, in some embodiments, when generating abnormal description information based on the result difference data, an abnormal alarm template may be obtained, the result difference data may be filled into the abnormal alarm template, and one or more fourth variables to be filled in the abnormal alarm template may be filled to obtain an abnormal alarm template to be instantiated. The abnormal alarm template to be instantiated is then instantiated to obtain abnormal description information. Thus, the generation efficiency of abnormal description information can be effectively improved based on the abnormal alarm template, and the accuracy of the obtained abnormal description information can be guaranteed.

[0183] Among them, the exception warning template refers to the pre-configured template used to generate exception description information.

[0184] The fourth variable to be filled refers to the variable that is pre-configured in the exception warning template.

[0185] Among them, the exception alarm template to be instantiated refers to the template obtained after the exception alarm template is filled with exception description information.

[0186] For example, such as Figure 10 As shown, Figure 10 This is a flowchart illustrating the processing of anomaly description information as presented in this disclosure, including the following steps:

[0187] (1) Periodically scan the verification results in the test database to identify discrepancies.

[0188] (2) Render the difference data of the verification results according to the alarm template to generate an anomaly description information, and send it to the relevant maintenance personnel via email.

[0189] In other words, in this embodiment of the present disclosure, after obtaining the execution result data of the billing system performing batch pricing tests on the verification test call detail records based on the verification test user data, it can be determined whether the execution result data and the result verification data meet the comparison rule indicators. If the execution result data and the result verification data meet the comparison rule indicators, it is determined that the batch pricing test result indicates that the billing system meets the test indicators. If the execution result data and the result verification data do not meet the comparison rule indicators, it is determined that the batch pricing test result indicates that the billing system does not meet the test indicators. Thus, it is possible to quickly and accurately determine whether the billing system meets the test indicators based on the comparison rule indicators, which can effectively improve the efficiency of batch pricing tests.

[0190] In this embodiment, after obtaining the execution result data of the billing system's batch pricing test on the verification test call detail records based on the verification test user data, it is determined whether the execution result data and the result verification data meet the comparison rule indicators. If the execution result data and the result verification data meet the comparison rule indicators, it is determined that the batch pricing test result indicates that the billing system meets the test indicators; if the execution result data and the result verification data do not meet the comparison rule indicators, it is determined that the batch pricing test result indicates that the billing system does not meet the test indicators. Thus, based on the comparison rule indicators, it is possible to quickly and accurately determine whether the billing system meets the test indicators, which can effectively improve the efficiency of batch pricing testing. By generating anomaly description information based on the execution result data and the result verification data, the anomaly description information is used to describe the situation in which the billing system generates anomalies. This provides anomaly description information, which can be generated and provided in a timely manner after it is determined that the batch pricing test result indicates that the billing system does not meet the test indicators, so as to quickly locate the anomaly point and perform fault analysis and repair. By determining the duration of the last batch test result of the billing system, reaching the first time interval, the result difference data between the execution result data and the result verification data is determined. Based on the result difference data, anomaly description information is generated. This allows for periodic scanning of the result difference data, effectively improving the robustness of the billing system. By obtaining an anomaly alarm template, the result difference data is filled into the anomaly alarm template, along with one or more fourth variables to be filled, to obtain the anomaly alarm template to be instantiated. The anomaly alarm template to be instantiated is then instantiated to obtain the anomaly description information. This approach effectively improves the generation efficiency of anomaly description information based on the anomaly alarm template while ensuring the accuracy of the obtained anomaly description information. After confirming that the first condition is met, a preset flag is added to the verification test call detail record (CDR). The preset flag is used to instruct the billing system to perform batch pricing tests on the verification test CDR and to instruct that the verification test CDR should not be consolidated. The billing system is provided with the verification test user information and the verification test CDR with the preset flag added. Thus, the billing system can be instructed to perform batch pricing tests on the verification test CDR based on the preset flag, without consolidating the verification test CDR, thereby effectively improving the rationality of the batch pricing test process and effectively avoiding the waste of computing resources.

[0191] For example, such as Figure 11 As shown, Figure 11 This is a schematic diagram of the verification and testing process for the billing system proposed in this disclosure, which includes the following steps:

[0192] (1) After the verification test automation tool is started, it automatically cleans up the user information, verification test call details, and result verification data of the previous verification test.

[0193] (2) The verification test automation tool provides templates for generating verification test sample data, including user information data templates, verification call detail record data templates, and result verification data templates;

[0194] (3) Pre-set the mobile phone number range of the verification test users (number range starting with "2"), which is used to generate verification test sample data according to the verification test data template for automated verification;

[0195] (4) The verification test can be automatically triggered by setting a periodic execution time, and the verification test can also be manually triggered in real time through the interface.

[0196] (5) Load the generated verification test sample data into the billing system business side database;

[0197] (6) The billing system performs business system verification tests based on the verification test user information and verification test call details. After the billing system completes the execution, it generates a result verification data file from the execution results of the verification test user's verification test call details.

[0198] (7) The verification test automation tool loads the result verification data file into the database and compares it with the result verification data according to the comparison rules. If the result data executed by the billing system is consistent with the result verification data, it means that there is no abnormality in the business system. If the two data are inconsistent, it means that there is an abnormality in the business system.

[0199] (8) Send the detailed information of the exceptions collected by the verification test automation tool (and the above exception description information) to relevant personnel such as business maintenance via email.

[0200] It is understood that the billing system batch testing method proposed in this disclosure can be implemented based on the automated verification system of the billing production system, such as... Figure 12 As shown, Figure 12 This is a structural diagram of the automated verification system for the billing production system proposed in this disclosure. The automated verification system is a separate production environment. An automated verification platform is built in the production environment to generate production verification test cases every day and execute them automatically. This realizes the automated error checking function of the billing rules in the production environment, reduces the probability of errors, and improves accuracy.

[0201] The modifications to the automated verification system include the following: scenario management (call detail record (CDR) sample generation, order material generation), verification data management (cumulative volume, balance, deduplication, list, reminders, credit control, rate limiting, and deduction), result display, and monitoring and alarms.

[0202] (1) Verification data generation: Based on the test number and related test data provided by the maintenance personnel, generate sample data of call detail records and order materials according to the call detail record template and the order material template, and at the same time generate verification test cases and target result data;

[0203] (2) Automatic execution: The automatic execution framework performs the calculation of the entire process according to the generated verification test cases based on sample data such as call detail records and order information, and stores the calculation results of each stage into the database respectively;

[0204] (3) Data verification: The calculation results of the automatic execution are compared with the target results for verification, and the verification results are stored in the database. Data verification includes cumulative amount, balance, deduplication, list, reminder, credit control, speed limit, deduction, etc.

[0205] (4) Monitoring and alarm: For data with inconsistent verification results, the monitoring and alarm application will send alarm information to relevant personnel via SMS according to the monitoring threshold and alarm rules.

[0206] Automated verification is primarily used during routine deployments or project cutovers to verify the correctness of project functions, assisting operations and maintenance personnel in identifying potential problems in advance and improving operational efficiency. For example... Figure 13 As shown, Figure 13 This is a flowchart illustrating the automated verification process proposed in this disclosure, wherein:

[0207] (1) Use case generation: Based on the verification data template, generate one use case for each package and each scenario, including user information, verification call detail records, and result verification data;

[0208] (2) Execute billing batch pricing: The production billing batch pricing application reads the verification test case data and executes the batch pricing of the verification call detail record;

[0209] (3) Saving of batch pricing results: The billing batch pricing results are stored in the production cumulative quantity table;

[0210] (4) Result verification: The execution engine of the automated verification tool audits and compares the batch price result with the verification result.

[0211] Figure 14 This is a schematic diagram of the structure of a pricing test device for a billing system according to an embodiment of this disclosure.

[0212] like Figure 14 As shown, the batch pricing test apparatus 140 for the billing system includes:

[0213] The generation module 14001 is used to generate verification test sample data, which includes: verification test user information, verification test call detail records and result verification data.

[0214] The first determining module 14002 is used to determine that the first condition is met and to provide the billing system with the verification test user information and the verification test call detail record.

[0215] Module 14003 is used to acquire the execution result data of the billing system's batch pricing test on the verification test call detail records based on the verification test user data; and

[0216] The second determining module 14004 is used to determine the batch pricing test results of the billing system based on the result verification data and execution result data.

[0217] It should be noted that the foregoing explanation of the batch pricing test method for the billing system also applies to the batch pricing test device for the billing system in this embodiment, and will not be repeated here.

[0218] In this embodiment, by generating verification test sample data, which includes verification test user information, verification test call detail records (CDRs), and result verification data, and determining that the first condition is met, the verification test user information and verification test CDRs are provided to the billing system. The execution result data of the billing system's batch pricing test on the verification test CDRs based on the verification test user information is obtained. Based on the result verification data and the execution result data, the batch pricing test result of the billing system is determined. This effectively improves the verification accuracy of the batch pricing correctness verification result and effectively reduces the verification time and workload required for verification.

[0219] Figure 15 A block diagram of an exemplary computer device suitable for implementing embodiments of the present disclosure is shown. Figure 15 The computer device 12 shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments disclosed herein.

[0220] like Figure 15 As shown, the computer device 12 is represented in the form of a general-purpose computing device. The components of the computer device 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and a bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0221] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. Examples of these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0222] Computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0223] Memory 28 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM) 30 and / or cache memory 32. Computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (…). Figure 15 Not shown; usually referred to as a "hard drive".

[0224] although Figure 15 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disc drive for reading and writing to a removable non-volatile optical disc (e.g., a compact disc read-only memory (CD-ROM), a digital video disc read-only memory (DVD-ROM), or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this disclosure.

[0225] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of this disclosure.

[0226] Computer device 12 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable human interaction with the computer device 12, and / or with any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 22. Furthermore, computer device 12 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 20. As shown, network adapter 20 communicates with other modules of computer device 12 via bus 18. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with computer device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0227] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the batch pricing test method for the billing system mentioned in the foregoing embodiments.

[0228] To implement the above embodiments, this disclosure also proposes a non-transitory computer-readable storage medium storing a computer program that, when executed by a processor, implements the batch pricing test method for a billing system as proposed in the foregoing embodiments of this disclosure.

[0229] To implement the above embodiments, this disclosure also proposes a computer program product that, when executed by an instruction processor, performs a batch pricing test method for a billing system as proposed in the foregoing embodiments of this disclosure.

[0230] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0231] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

[0232] It should be noted that in the description of this disclosure, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this disclosure, unless otherwise stated, "a plurality of" means two or more.

[0233] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of preferred embodiments of this disclosure includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the function involved, as will be understood by those skilled in the art to which embodiments of this disclosure pertain.

[0234] It should be understood that various parts of this disclosure can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0235] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it includes one or a combination of the steps of the method embodiments.

[0236] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0237] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.

[0238] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0239] Although embodiments of the present disclosure have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present disclosure.

Claims

1. A batch pricing test method for a billing system, characterized in that, The method includes: Generate verification test sample data, wherein the verification test sample data includes: verification test user information, verification test call detail records and result verification data; Once the first condition is met, a preset mark is added to the verification test call detail record (CDR). The preset mark is used to instruct the billing system to perform batch pricing tests on the CDR and to instruct that the CDR should not be consolidated. Provide the billing system with the verification test user information and the verification test call detail records after adding the preset tags; Obtain the execution result data of the billing system performing batch pricing tests on the verification test call details based on the verification test user data; and Based on the verification data and the execution result data, the batch pricing test results of the billing system are determined; The step of determining the batch pricing test result of the billing system based on the verification data and the execution result data includes: Determine whether the execution result data and the result verification data meet the comparison rule indicators; If the execution result data and the result verification data meet the comparison rule indicators, then the batch pricing test result indicates that the billing system meets the test indicators; If the execution result data and the result verification data do not meet the comparison rule indicators, then the batch pricing test result indicates that the billing system does not meet the test indicators; After determining that the batch pricing test result indicates that the billing system does not meet the test indicators, the duration since the last determination of the batch pricing test result of the billing system reaches a first time interval. Determine the result difference data between the execution result data and the result verification data; Get the abnormal alarm template; The difference data of the results are filled into the abnormal alarm template, and one or more fourth variables to be filled in the abnormal alarm template are filled in to obtain the abnormal alarm template to be instantiated. The abnormal alarm template to be instantiated is instantiated to obtain abnormal description information, wherein the abnormal description information is used to describe the abnormal situation of the billing system. Provide the description information of the anomaly; The generation of verification test sample data includes: Based on the virtual number information and the verification test data template, the verification test sample data is generated, wherein the virtual number information and the real number information are different; The verification test data template includes at least one of the following: User profile data template; Verify call detail record (CDR) data template; Result verification data template; Before generating the verification test sample data based on the virtual number information and the verification test data template, the method further includes: Delete the verification test user information associated with the virtual number information used in the previous test from the business database; Remove the verification data from the verification test database that is associated with the virtual number information used in the previous test; Delete the execution result data associated with the virtual number information used in the previous test from the billing system's database; Delete the verification test user information, verification test call details, and result verification data associated with the virtual number information used in the previous test from local storage.

2. The method as described in claim 1, characterized in that, The step of generating the verification test sample data based on the virtual number information and the verification test data template includes: The verification test user profile is generated based on the virtual number information and the user profile data template; The verification test call detail record (CDR) is generated based on the virtual number information and the verification call detail record (CDR) data template. The result verification data is generated based on the virtual number information and the result verification data template.

3. The method as described in claim 2, characterized in that, The step of generating the verification test user data based on the virtual number information and the user data template includes: The virtual number information is filled into the user data template, and one or more first variables to be filled in the user data template are filled in to obtain the user data template to be instantiated. The user profile data template to be instantiated is instantiated to obtain the verification test user profile.

4. The method as described in claim 2, characterized in that, The step of generating the verification test call detail record (CDR) based on the virtual number information and the verification CDR data template includes: The virtual number information is filled into the verification call detail record (CDR) data template, and one or more second variables to be filled in the CDR data template are also filled in to obtain the CDR data template to be instantiated. The call detail record (CDR) data template to be instantiated is instantiated to obtain the verification test CDR.

5. The method as described in claim 2, characterized in that, The step of generating the result verification data based on the virtual number information and the result verification data template includes: The virtual number information is filled into the result verification data template, and one or more third variables to be filled in the result verification data template are filled in to obtain the result verification data template to be instantiated. The result verification data template to be instantiated is instantiated to obtain the result verification data.

6. The method as described in claim 1, characterized in that, The method further includes: The virtual number information and the verification test user information are associated and stored in the business database; The virtual number information and the result verification data are associated and stored in the verification test database; The virtual number information and the execution result data are associated and stored in the database of the billing system; The virtual number information, the verification test user information, the verification test call detail record, and the result verification data are associated and stored in local storage.

7. The method according to any one of claims 1-6, characterized in that, The virtual number information includes at least one of the following: Virtual number segments; Virtual number user identifier; Virtual International Mobile Subscriber Identity (IMSI).

8. The method according to any one of claims 1-6, characterized in that, The execution result data and the result verification data each include data from at least one of the following dimensions: Cumulative data, balance data, duplicate data, list data, reminder data, credit control data, speed limit data, and deduction data.

9. The method according to any one of claims 1-6, characterized in that, The determination that the first condition is met includes at least one of the following: The duration since the last execution of the verification test of the billing system has reached the second time interval. An execution instruction is received, wherein the execution instruction is used to trigger the execution of the verification test of the billing system.

10. A batch pricing test device for a billing system, characterized in that, The device includes: The generation module is used to generate verification test sample data, wherein the verification test sample data includes: verification test user information, verification test call detail records and result verification data; The first determining module is used to determine that a first condition is met, add a preset mark to the verification test call detail record (CDR), wherein the preset mark is used to instruct the billing system to perform batch pricing testing on the verification test CDR, and to instruct that the verification test CDR should not be consolidated; and to provide the billing system with the verification test user information and the verification test CDR after adding the preset mark; The acquisition module is used to acquire the execution result data of the billing system's batch pricing test on the verification test call detail records based on the verification test user information; and The second determining module is used to determine the batch pricing test result of the billing system based on the result verification data and the execution result data; The second determining module is specifically used for: determining whether the execution result data and the result verification data meet the comparison rule indicators; if the execution result data and the result verification data meet the comparison rule indicators, then determining that the batch pricing test result indicates that the billing system meets the test indicators; if the execution result data and the result verification data do not meet the comparison rule indicators, then determining that the batch pricing test result indicates that the billing system does not meet the test indicators; after determining that the batch pricing test result indicates that the billing system does not meet the test indicators, determining that the duration since the last determination of the batch pricing test result of the billing system has reached a first time interval; determining the result difference data between the execution result data and the result verification data; obtaining an abnormal alarm template; filling the result difference data into the abnormal alarm template, and filling one or more fourth variables to be filled in the abnormal alarm template to obtain an abnormal alarm template to be instantiated; instantiating the abnormal alarm template to be instantiated to obtain abnormal description information, wherein the abnormal description information is used to describe the situation in which the billing system generates an abnormality; and providing the abnormal description information. The generation module is used for: Based on the virtual number information and the verification test data template, the verification test sample data is generated, wherein the virtual number information and the real number information are different; The verification test data template includes at least one of the following: User profile data template; Verify call detail record (CDR) data template; Result verification data template; Before generating the verification test sample data based on the virtual number information and the verification test data template, the device also performs the following: Delete the verification test user information associated with the virtual number information used in the previous test from the business database; Remove the verification data from the verification test database that is associated with the virtual number information used in the previous test; Delete the execution result data associated with the virtual number information used in the previous test from the billing system's database; Delete the verification test user information, verification test call details, and result verification data associated with the virtual number information used in the previous test from local storage.

11. A computer device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-9.

12. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, in, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-9.

13. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1-9.

Images