Big data comparison test method, device, electronic device and storage medium

Abstract

The present invention relates to the field of automated testing technology, and provides a big data comparison test method, device, electronic device and storage medium, which collects transaction data and transaction messages in a target partition in a production environment, and desensitizes the transaction data and transaction messages based on the historical data of the target partition to obtain target transaction data and target transaction messages, synchronizes the target transaction data and target transaction messages to a test environment, removes redundancy from the target transaction messages according to the target transaction data, and generates a test case set according to the target transaction messages after redundancy removal, and finally executes the test case set in two different versions of the test environment to obtain two test results, and outputs the comparison result after comparing the two test results. The present invention covers the full amount of transaction scenarios in the production environment by testing and comparing the transaction data and transaction messages in the production environment in the test environment, thereby improving the accuracy of the test.

Description

Technical Field

[0001] The present invention relates to the field of automated testing technology, and in particular to a big data comparison test method, device, electronic equipment and storage medium. Background Art

[0002] In the process of system development and application, in order to ensure the correct and smooth operation of the system, developers often build an automated testing environment and perform simulation tests in the automated testing environment to determine whether the system can run correctly in the production environment.

[0003] The inventors found that in production, the total transaction volume and authorized full transaction codes for each authorized transaction day are as high as tens of millions, and the automated cases cannot simulate real transaction data and transaction messages, resulting in poor test accuracy. Summary of the invention

[0004] In view of the above, it is necessary to propose a big data comparison test method, device, electronic device and storage medium, which improves the accuracy of the test by testing and comparing transaction data and transaction messages in a production environment in a test environment.

[0005] A first aspect of the present invention provides a big data comparison test method, the method comprising:

[0006] Determine the target partition in the production environment, and collect transaction data and transaction messages in the target partition;

[0007] Desensitizing the transaction data based on the historical data of the target partition to obtain target transaction data, and desensitizing the transaction message to obtain a target transaction message;

[0008] Synchronizing the target transaction data and the target transaction message into the test environment, and removing redundancy from the target transaction message according to the target transaction data;

[0009] Generate a test case set based on the target transaction message after redundancy removal;

[0010] Execute the test case set in a preset first version test environment to obtain a first test result, and execute the test case set in a preset second version test environment to obtain a second test result;

[0011] The first test result and the second test result are compared, and the comparison result is output.

[0012] According to an optional implementation manner of the present invention, the collecting the transaction data and transaction messages in the target partition includes:

[0013] Obtain the acquisition control timing information of the data acquisition configuration process;

[0014] Matching each timing node in the acquisition control timing information with a preset timing allocation program to obtain process allocation control information;

[0015] The transaction data and transaction messages in the target partition are collected according to the process allocation control information.

[0016] According to an optional implementation manner of the present invention, the acquiring of the acquisition control timing information of the data acquisition configuration process includes:

[0017] Dividing the data acquisition configuration process into a plurality of unrelated configuration partition objects;

[0018] Determine a partition service label of the configuration partition object;

[0019] Extracting multiple groups of information from the data collection configuration process;

[0020] The acquisition control timing information of the data acquisition configuration process is obtained according to each group of information extraction results and the partition service tag of each of the configured partition objects.

[0021] According to an optional implementation of the present invention, the historical data of the target partition includes historical transaction data and historical transaction messages, and the desensitizing the transaction data based on the historical data of the target partition to obtain the target transaction data, and the desensitizing the transaction message to obtain the target transaction message includes:

[0022] Desensitizing the historical transaction data using multiple first desensitizing algorithms to obtain first desensitized data corresponding to each of the first desensitizing algorithms, and desensitizing the historical transaction messages using multiple second desensitizing algorithms to obtain second desensitized data corresponding to each of the second desensitizing algorithms;

[0023] Calculate a first desensitization effect of the first desensitization algorithm according to the historical transaction data and the first desensitization data, and calculate a second desensitization effect of the second desensitization algorithm according to the historical transaction message and the second desensitization data;

[0024] Determine a first target desensitization algorithm from the multiple first desensitization algorithms according to the first desensitization effect, and use the first target desensitization algorithm to perform desensitization processing on the transaction data to obtain target transaction data;

[0025] A second target desensitization algorithm is determined from the multiple second desensitization algorithms according to the second desensitization effect, and the transaction message is desensitized using the second target desensitization algorithm to obtain a target transaction message.

[0026] According to an optional implementation of the present invention, the calculating, according to the historical transaction data and the first desensitized data, a first desensitization effect of the first desensitization algorithm corresponding to the first desensitization data includes:

[0027] For any one of the first desensitizing algorithms, obtaining the number of fields that need to be desensitized in the historical transaction data and obtaining the number of fields that have been desensitized in the first desensitized data;

[0028] The desensitization coverage rate is calculated according to the number of fields that need to be desensitized and the number of fields that have been desensitized;

[0029] Verify the fields that need to be desensitized and the fields that have been desensitized to obtain the number of successfully desensitized fields;

[0030] The desensitization success rate is calculated according to the number of successful desensitization and the number of fields that need to be desensitized;

[0031] The first desensitization effect is calculated according to the desensitization coverage rate and the desensitization success rate.

[0032] According to an optional implementation manner of the present invention, the step of removing redundancy from the target transaction message according to the target transaction data includes:

[0033] Obtaining the target logical address and target message sequence number carried in the target transaction message;

[0034] Read the historical logical address carried in the historical transaction message and the historical message sequence number corresponding to the historical logical address;

[0035] Performing initial redundancy removal on the target transaction message according to the historical message sequence number and the target message sequence number;

[0036] Determining a transaction channel of the target transaction data, and classifying the target transaction data according to the transaction channel;

[0037] Redundancy removal is performed on each type of target transaction data, and secondary redundancy removal is performed on the target transaction message after the initial redundancy removal according to the target transaction data after the redundancy removal.

[0038] According to an optional implementation manner of the present invention, generating a test case set according to the target transaction message after redundancy removal includes:

[0039] Obtaining a message header label of the target transaction message after the redundancy is removed;

[0040] Obtaining a response message according to the message header label;

[0041] A test case set is generated according to the target transaction message after redundancy removal and the response message.

[0042] A second aspect of the present invention provides a big data comparison test device, the device comprising:

[0043] A collection module, used to determine a target partition in a production environment and collect transaction data and transaction messages in the target partition;

[0044] A desensitizing module, configured to perform desensitizing processing on the transaction data based on the historical data of the target partition to obtain target transaction data, and perform desensitizing processing on the transaction message to obtain a target transaction message;

[0045] A redundancy removal module, used to synchronize the target transaction data and the target transaction message to the test environment, and remove redundancy from the target transaction message according to the target transaction data;

[0046] A generation module, used to generate a test case set according to the target transaction message after redundancy removal;

[0047] An execution module, configured to execute the test case set in a preset first version of the test environment to obtain a first test result, and execute the test case set in a preset second version of the test environment to obtain a second test result;

[0048] The comparison module is used to compare the first test result and the second test result, and output the comparison result.

[0049] A third aspect of the present invention provides an electronic device, comprising a processor, wherein the processor is configured to implement the big data comparison test method when executing a computer program stored in a memory.

[0050] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, and the computer program implements the big data comparison test method when executed by a processor.

[0051] To summarize, the big data comparison test method, device, electronic device and storage medium described in the present invention, after collecting the transaction data and transaction messages in the target partition in the production environment, in order to ensure the safe use of the transaction data and transaction messages in the customer's production environment in the non-production environment and prevent the leakage of sensitive information, firstly desensitizes the transaction data and transaction messages based on the historical data of the target partition to obtain the target transaction data and target transaction messages, and then synchronizes the target transaction data and target transaction messages to the test environment, so as to realize the simulation of the full transaction scenarios in the real production environment in the test environment. In order to improve the testing efficiency of the target transaction data and target transaction messages in the test environment, the target transaction messages are de-redundant according to the target transaction data, and a test case set is generated according to the target transaction messages after the de-redundancy is removed. Finally, the test case set is executed in two different versions of the test environment to obtain two test results, and the comparison result is output after the two test results are compared, thereby improving the accuracy of the test. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is a flow chart of the big data comparison test method provided in Example 1 of the present invention.

[0053] Figure 2 It is a structural diagram of a big data comparison test device provided in Example 2 of the present invention.

[0054] Figure 3 It is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present invention. DETAILED DESCRIPTION

[0055] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.

[0056] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing embodiments in an optional embodiment and are not intended to limit the present invention.

[0057] The big data comparison test method provided in the embodiment of the present invention is executed by an electronic device, and accordingly, the big data comparison test device runs in the electronic device.

[0058] Embodiment 1

[0059] Figure 1This is a flow chart of a big data comparison test method provided by Embodiment 1 of the present invention. The big data comparison test method specifically includes the following steps. According to different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted.

[0060] S11, determining a target partition in a production environment, and collecting transaction data and transaction messages in the target partition.

[0061] The production environment refers to the environment in which the application or device is ultimately used, including server hardware, software, network equipment, etc.

[0062] The transaction data produced in the production environment is stored in the database, and the database is distributed, so the transaction data is also stored in partitions. For example, the production environment can produce tens of millions of transaction data and corresponding transaction messages in a day, which are distributed and stored in dozens of partitions.

[0063] Each partition in the production environment is set with a region ID. The target partition can be specified by the user (for example, developer, tester) or randomly selected.

[0064] After the target partition is determined, the transaction data and transaction messages of the target partition may be collected, and the transaction data and transaction messages may be sent to the test environment for testing.

[0065] In an optional implementation, the collecting the transaction data and transaction messages in the target partition includes:

[0066] Obtain the acquisition control timing information of the data acquisition configuration process;

[0067] Matching each timing node in the acquisition control timing information with a preset timing allocation program to obtain process allocation control information;

[0068] The transaction data and transaction messages in the target partition are collected according to the process allocation control information.

[0069] The data collection configuration process may be preset. The collection control timing information includes a plurality of timing nodes, and each timing node is matched with a preset timing allocation program, thereby associating the timing node with the timing allocation program to obtain process allocation control information.

[0070] This optional implementation method obtains the acquisition control timing information of the data acquisition configuration process, and obtains the process allocation control information according to the matching relationship between each timing node in the acquisition control timing information and the preset timing allocation program, and then collects the transaction data and transaction messages in the target partition according to the process allocation control information. In this way, the depth information of the acquisition control timing in the data acquisition process is taken into account, thereby greatly reducing a series of inaccurate acquisition control situations that may occur in the data acquisition process, and improving the accuracy of the acquisition control results.

[0071] In an optional implementation, the acquiring of the acquisition control timing information of the data acquisition configuration process includes:

[0072] Dividing the data acquisition configuration process into a plurality of unrelated configuration partition objects;

[0073] Determine a partition service label of the configuration partition object;

[0074] Extracting multiple groups of information from the data collection configuration process;

[0075] The acquisition control timing information of the data acquisition configuration process is obtained according to each group of information extraction results and the partition service tag of each of the configured partition objects.

[0076] The strategy tree information extraction method can be used to perform feature separation processing on the data collection configuration process, and information can be extracted based on the feature separation processing results.

[0077] S12, performing desensitization processing on the transaction data based on the historical data of the target partition to obtain target transaction data, and performing desensitization processing on the transaction message to obtain a target transaction message.

[0078] Before sending transaction data and transaction messages to the test environment, in order to ensure that the transaction data and transaction messages in the customer's production environment can be safely used in the non-production environment, prevent sensitive information leakage, and ensure data security, the transaction data and transaction messages are first desensitized.

[0079] Sensitive fields in transaction data and transaction messages can be determined, and then a desensitizing algorithm can be used to desensitize the sensitive fields. The same desensitizing algorithm can be used to desensitize all sensitive fields, or different desensitizing algorithms can be used to desensitize different sensitive fields. The present invention does not impose any limitations.

[0080] In an optional implementation, the historical data of the target partition includes historical transaction data and historical transaction messages, and the desensitizing the transaction data based on the historical data of the target partition to obtain the target transaction data, and desensitizing the transaction message to obtain the target transaction message includes:

[0081] Desensitizing the historical transaction data using multiple first desensitizing algorithms to obtain first desensitized data corresponding to each of the first desensitizing algorithms, and desensitizing the historical transaction messages using multiple second desensitizing algorithms to obtain second desensitized data corresponding to each of the second desensitizing algorithms;

[0082] Calculate a first desensitization effect of the first desensitization algorithm according to the historical transaction data and the first desensitization data, and calculate a second desensitization effect of the second desensitization algorithm according to the historical transaction message and the second desensitization data;

[0083] Determine a first target desensitization algorithm from the multiple first desensitization algorithms according to the first desensitization effect, and use the first target desensitization algorithm to perform desensitization processing on the transaction data to obtain target transaction data;

[0084] A second target desensitization algorithm is determined from the multiple second desensitization algorithms according to the second desensitization effect, and the transaction message is desensitized using the second target desensitization algorithm to obtain a target transaction message.

[0085] A plurality of first desensitizing algorithms and a plurality of second desensitizing algorithms are pre-stored in the big data testing platform, wherein each first desensitizing algorithm is used to desensitize transaction data, and each second desensitizing algorithm is used to desensitize transaction messages. The present invention will not list the specific first desensitizing algorithms and the second desensitizing algorithms one by one. The present invention mainly evaluates the desensitizing effect of the first desensitizing algorithm after desensitizing the transaction data by the first desensitizing algorithm, thereby determining the first target desensitizing algorithm for desensitizing the target transaction data according to the desensitizing effect; and evaluates the desensitizing effect of the second desensitizing algorithm after desensitizing the transaction message by the second desensitizing algorithm, thereby determining the second target desensitizing algorithm for desensitizing the target transaction message according to the desensitizing effect.

[0086] Generally speaking, the transaction data and transaction messages stored in the target partition are identical in terms of field types and other aspects. Therefore, the current target transaction data can be desensitized based on the desensitizing effect of historical transaction data, and the current target transaction message can be desensitized based on the desensitizing effect of historical transaction messages.

[0087] The above optional implementation method desensitizes historical transaction data and transaction messages through a desensitizing algorithm to obtain desensitized historical transaction data and historical transaction messages, thereby calculating the desensitizing effect of each desensitizing algorithm based on the historical transaction data and historical transaction messages before and after desensitization, and then determining the target desensitizing algorithm from the desensitizing algorithm according to the desensitizing effect, and finally using the target desensitizing algorithm to desensitize the target transaction data and target transaction messages, thereby improving the desensitizing effect of the target transaction data and target transaction messages, ensuring the safe use of customers' transaction data and transaction messages in production environments in non-production environments, preventing the leakage of sensitive information, and ensuring data security.

[0088] In an optional implementation, the calculating, according to the historical transaction data and the first desensitized data, a first desensitization effect of the first desensitization algorithm corresponding to the first desensitization data includes:

[0089] For any one of the first desensitizing algorithms, obtaining the number of fields that need to be desensitized in the historical transaction data and obtaining the number of fields that have been desensitized in the first desensitized data;

[0090] The desensitization coverage rate is calculated according to the number of fields that need to be desensitized and the number of fields that have been desensitized;

[0091] Verify the fields that need to be desensitized and the fields that have been desensitized to obtain the number of successfully desensitized fields;

[0092] The desensitization success rate is calculated according to the number of successful desensitization and the number of fields that need to be desensitized;

[0093] The first desensitization effect is calculated according to the desensitization coverage rate and the desensitization success rate.

[0094] Desensitization coverage = number of fields that need to be desensitized / number of fields that have been desensitized. Desensitization coverage is used to describe the coverage of the desensitization algorithm for the desensitized data. It can intuitively reflect whether the desensitization algorithm has the ability to process various typical data. The higher the desensitization coverage, the better the desensitization effect, and the lower the desensitization coverage, the worse the desensitization effect.

[0095] Desensitization success rate = number of successfully desensitized fields / number of fields that need to be desensitized. The desensitization success rate is used to characterize the record of the successful desensitization of the data in the data set by the desensitization algorithm. The higher the desensitization success rate, the better the desensitization effect, and the lower the desensitization success rate, the worse the desensitization effect.

[0096] Whether the field to be desensitized is successfully desensitized can be determined by verifying whether the length of the field to be desensitized is consistent with the length of the field that has been desensitized. If the lengths are consistent, it is determined that the desensitization is successful. If the lengths are inconsistent, it is determined that the desensitization has failed. It is also possible to determine whether the field to be desensitized is successfully desensitized by verifying whether the field value to be desensitized is the same as the field value that has been desensitized. If the field values ​​are the same, it is determined that the desensitization is successful. If the field values ​​are different, it is determined that the desensitization has failed.

[0097] The above optional implementation method calculates the desensitization coverage and desensitization success rate through the transaction data or transaction messages before and after desensitization, and combines the two dimensions of desensitization coverage and desensitization success rate to jointly determine the desensitization effect of each desensitization algorithm. The calculation accuracy of the desensitization effect is higher, which helps to determine the target desensitization algorithm, and then improve the desensitization quality of the target transaction data or target transaction messages.

[0098] S13, synchronizing the target transaction data and the target transaction message into a test environment, and removing redundancy from the target transaction message according to the target transaction data.

[0099] The test environment refers to the server hardware, software, network equipment, and historical data required to complete the test work.

[0100] The desensitized transaction data and transaction messages in the production environment are synchronized to the test environment for testing, so that the full range of transaction scenarios in the real production environment can be simulated in the test environment.

[0101] In order to improve the test efficiency of the target transaction data and the target transaction message in the test environment, it is necessary to remove redundancy from the target transaction data and the target transaction message.

[0102] In an optional implementation, the removing redundancy from the target transaction message according to the target transaction data includes:

[0103] Obtaining the target logical address and target message sequence number carried in the target transaction message;

[0104] Read the historical logical address carried in the historical transaction message and the historical message sequence number corresponding to the historical logical address;

[0105] Performing initial redundancy removal on the target transaction message according to the historical message sequence number and the target message sequence number;

[0106] Determining a transaction channel of the target transaction data, and classifying the target transaction data according to the transaction channel;

[0107] Redundancy removal is performed on each type of target transaction data, and secondary redundancy removal is performed on the target transaction message after the initial redundancy removal according to the target transaction data after the redundancy removal.

[0108] The logical address may be a logical address corresponding to a terminal of a business system corresponding to a production environment, and the message sequence number is a cyclically increasing sequence.

[0109] The historical message sequence numbers corresponding to the historical logical addresses carried by the historical transaction messages in the target partition are collected together to obtain a historical message sequence number set, and each target message sequence number in the target transaction message is matched one by one with each historical message sequence number in the historical message sequence number set. When the target message sequence number matches the historical message sequence number successfully, the target transaction message corresponding to the target message sequence number is retained, and when the target message sequence number fails to match the historical message sequence number, the target transaction message corresponding to the target message sequence number is deleted. By matching the historical message sequence number with the target message sequence number, the target transaction message corresponding to the target message sequence number that fails to match is deleted, thereby realizing the initial redundancy removal of the target transaction message.

[0110] In the test environment, the target transaction data can be classified according to the transaction channels to obtain multiple types of target transaction data. The transaction channels may include: UnionPay, ChinaNet, ChinaNet, MasterCard, MasterNet, Liantong, etc.

[0111] Some classes have a lot of target transaction data, while some classes have very little target transaction data. For the target transaction data of the class with a small number, all of them are retained. For the target transaction data of the class with a large number, some of the target transaction data can be selected for testing based on the target transaction data of the class with a small number. It is also possible to select some of the target transaction data from the target transaction data of the class with a large number for testing according to a preset ratio. After the target transaction data is de-redundant, it must be ensured that the target transaction message corresponds to the target transaction data one by one, so that a test case set can be generated based on the target transaction data and the target transaction message after de-redundancy.

[0112] The above optional implementation method performs initial redundancy removal on the target transaction message through the logical address and the message sequence number, which can avoid redundancy failure caused by different delays in transmitting transaction messages through different channels or changes in the message sequence caused by forwarding devices such as switches; classifying the target transaction data according to the transaction channel and performing redundancy removal can effectively ensure that the transaction data of each type of transaction channel can participate in the test; performing secondary redundancy removal on the target transaction message after the initial redundancy removal according to the target transaction data after the redundancy removal can effectively ensure that the target transaction data after the redundancy removal corresponds one-to-one with the target transaction message, which is convenient for the subsequent generation of a test case set; in addition, the number of target transaction data after redundancy removal is greatly reduced compared with the target transaction data before the redundancy removal, reducing the number of target transaction data tested is conducive to saving test time and improving test efficiency.

[0113] S14, generating a test case set according to the target transaction message after redundancy removal.

[0114] A test case set is a set of test inputs, execution conditions, and expected results compiled for a specific purpose in order to test a program path or verify whether a specific requirement is met.

[0115] In a test environment, you can manually call a self-developed tool to read the message in the database table and resend the message request to the authorized application.

[0116] In an optional implementation, generating a test case set according to the target transaction message after redundancy removal includes:

[0117] Obtaining a message header label of the target transaction message after the redundancy is removed;

[0118] Obtaining a response message according to the message header label;

[0119] A test case set is generated according to the target transaction message after redundancy removal and the response message.

[0120] The corresponding response message can be obtained from the production environment according to the message header label, so that the target transaction message after redundancy removal and the response message are used as a test case set, and the test case set is stored in the test library for repeated use.

[0121] S15, executing the test case set in a preset first version test environment to obtain a first test result, and executing the test case set in a preset second version test environment to obtain a second test result.

[0122] The preset first version test environment refers to an environment that is the same as the actual production environment, and the preset second version test environment refers to an environment that is higher than the actual production environment.

[0123] Use two different versions of the test environment to test the test case set at the same time, and replay it on the benchmark machine after obtaining the test results. Obtain the response message after the test for full comparison to show the test differences between different test environments.

[0124] S16, comparing the first test result and the second test result, and outputting the comparison result.

[0125] The comparison result may include: the first test result is the same as the second test result, and the first test result is different from the second test result.

[0126] When the first test result is the same as the second test result, it indicates that the actual production environment can run stably.

[0127] When the first test result is different from the second test result, it indicates that a problem occurs in the actual production environment, for example, a requirement change occurs or a system code is changed.

[0128] In an optional embodiment, the method further comprises:

[0129] When the comparison result shows that the first test result is different from the second test result, identifying difference data between the first test result and the second test result;

[0130] A test report is generated according to the difference data.

[0131] The test report can contain data comparison and display results.

[0132] The above optional implementation mode generates a test report and displays the difference data between the first test result and the second test result intuitively, which can facilitate developers or testers to find the cause of the difference based on the difference data, so as to quickly propose corresponding solutions.

[0133] The embodiments of the present invention can significantly improve the comprehensiveness, accuracy, coverage and testing efficiency of data testing, thereby ensuring the stable operation of the production environment and reducing the possibility of production accidents.

[0134] Embodiment 2

[0135] Figure 2 It is a structural diagram of a big data comparison test device provided in Example 2 of the present invention.

[0136] In some embodiments, the big data comparison test device 20 may include multiple functional modules composed of computer program segments. The computer programs of each program segment in the big data comparison test device 20 may be stored in a memory of an electronic device and executed by at least one processor to perform (see Figure 1 Description) Function of big data comparison test.

[0137] In this embodiment, the big data comparison test device 20 can be divided into multiple functional modules according to the functions performed. The functional modules may include: an acquisition module 201, a desensitization module 202, a residual removal module 203, a generation module 204, an execution module 205 and a comparison module 206. The module referred to in the present invention refers to a series of computer program segments that can be executed by at least one processor and can complete fixed functions, which are stored in a memory. In this embodiment, the functions of each module will be described in detail in subsequent embodiments.

[0138] The collection module 201 is used to determine the target partition in the production environment and collect transaction data and transaction messages in the target partition.

[0139] The production environment refers to the environment in which the application or device is ultimately used, including server hardware, software, network equipment, etc.

[0140] The transaction data produced in the production environment is stored in the database, and the database is distributed, so the transaction data is also stored in partitions. For example, the production environment can produce tens of millions of transaction data and corresponding transaction messages in a day, which are distributed and stored in dozens of partitions.

[0141] Each partition in the production environment is set with a region ID. The target partition can be specified by the user (for example, developer, tester) or randomly selected.

[0142] After the target partition is determined, the transaction data and transaction messages of the target partition may be collected, and the transaction data and transaction messages may be sent to the test environment for testing.

[0143] In an optional implementation, the acquisition module 201 acquires the transaction data and transaction messages in the target partition including:

[0144] Obtain the acquisition control timing information of the data acquisition configuration process;

[0145] Matching each timing node in the acquisition control timing information with a preset timing allocation program to obtain process allocation control information;

[0146] The transaction data and transaction messages in the target partition are collected according to the process allocation control information.

[0147] The data collection configuration process may be preset. The collection control timing information includes a plurality of timing nodes, and each timing node is matched with a preset timing allocation program, thereby associating the timing node with the timing allocation program to obtain process allocation control information.

[0148] This optional implementation method obtains the acquisition control timing information of the data acquisition configuration process, and obtains the process allocation control information according to the matching relationship between each timing node in the acquisition control timing information and the preset timing allocation program, and then collects the transaction data and transaction messages in the target partition according to the process allocation control information. In this way, the depth information of the acquisition control timing in the data acquisition process is taken into account, thereby greatly reducing a series of inaccurate acquisition control situations that may occur in the data acquisition process, and improving the accuracy of the acquisition control results.

[0149] In an optional implementation, the acquiring of the acquisition control timing information of the data acquisition configuration process includes:

[0150] Dividing the data acquisition configuration process into a plurality of unrelated configuration partition objects;

[0151] Determine a partition service label of the configuration partition object;

[0152] Extracting multiple groups of information from the data collection configuration process;

[0153] The acquisition control timing information of the data acquisition configuration process is obtained according to each group of information extraction results and the partition service tag of each of the configured partition objects.

[0154] The strategy tree information extraction method can be used to perform feature separation processing on the data collection configuration process, and information can be extracted based on the feature separation processing results.

[0155] The desensitizing module 202 is used to perform desensitizing processing on the transaction data based on the historical data of the target partition to obtain target transaction data, and to perform desensitizing processing on the transaction message to obtain a target transaction message.

[0156] Before sending transaction data and transaction messages to the test environment, in order to ensure that the transaction data and transaction messages in the customer's production environment can be safely used in the non-production environment, prevent sensitive information leakage, and ensure data security, the transaction data and transaction messages are first desensitized.

[0157] Sensitive fields in transaction data and transaction messages can be determined, and then a desensitizing algorithm can be used to desensitize the sensitive fields. The same desensitizing algorithm can be used to desensitize all sensitive fields, or different desensitizing algorithms can be used to desensitize different sensitive fields. The present invention does not impose any limitations.

[0158] In an optional implementation, the historical data of the target partition includes historical transaction data and historical transaction messages, and the desensitizing module 202 performs desensitizing processing on the transaction data based on the historical data of the target partition to obtain target transaction data, and performs desensitizing processing on the transaction message to obtain the target transaction message, including:

[0159] Desensitizing the historical transaction data using multiple first desensitizing algorithms to obtain first desensitized data corresponding to each of the first desensitizing algorithms, and desensitizing the historical transaction messages using multiple second desensitizing algorithms to obtain second desensitized data corresponding to each of the second desensitizing algorithms;

[0160] Calculate a first desensitization effect of the first desensitization algorithm according to the historical transaction data and the first desensitization data, and calculate a second desensitization effect of the second desensitization algorithm according to the historical transaction message and the second desensitization data;

[0161] Determine a first target desensitization algorithm from the multiple first desensitization algorithms according to the first desensitization effect, and use the first target desensitization algorithm to perform desensitization processing on the transaction data to obtain target transaction data;

[0162] A second target desensitization algorithm is determined from the multiple second desensitization algorithms according to the second desensitization effect, and the transaction message is desensitized using the second target desensitization algorithm to obtain a target transaction message.

[0163] A plurality of first desensitizing algorithms and a plurality of second desensitizing algorithms are pre-stored in the big data testing platform, wherein each first desensitizing algorithm is used to desensitize transaction data, and each second desensitizing algorithm is used to desensitize transaction messages. The present invention will not list the specific first desensitizing algorithms and the second desensitizing algorithms one by one. The present invention mainly evaluates the desensitizing effect of the first desensitizing algorithm after desensitizing the transaction data by the first desensitizing algorithm, thereby determining the first target desensitizing algorithm for desensitizing the target transaction data according to the desensitizing effect; and evaluates the desensitizing effect of the second desensitizing algorithm after desensitizing the transaction message by the second desensitizing algorithm, thereby determining the second target desensitizing algorithm for desensitizing the target transaction message according to the desensitizing effect.

[0164] Generally speaking, the transaction data and transaction messages stored in the target partition are identical in terms of field types and other aspects. Therefore, the current target transaction data can be desensitized based on the desensitizing effect of historical transaction data, and the current target transaction message can be desensitized based on the desensitizing effect of historical transaction messages.

[0165] The above optional implementation method desensitizes historical transaction data and transaction messages through a desensitizing algorithm to obtain desensitized historical transaction data and historical transaction messages, thereby calculating the desensitizing effect of each desensitizing algorithm based on the historical transaction data and historical transaction messages before and after desensitization, and then determining the target desensitizing algorithm from the desensitizing algorithm according to the desensitizing effect, and finally using the target desensitizing algorithm to desensitize the target transaction data and target transaction messages, thereby improving the desensitizing effect of the target transaction data and target transaction messages, ensuring the safe use of customers' transaction data and transaction messages in production environments in non-production environments, preventing the leakage of sensitive information, and ensuring data security.

[0166] In an optional implementation, the calculating, according to the historical transaction data and the first desensitized data, a first desensitization effect of the first desensitization algorithm corresponding to the first desensitization data includes:

[0167] For any one of the first desensitizing algorithms, obtaining the number of fields that need to be desensitized in the historical transaction data and obtaining the number of fields that have been desensitized in the first desensitized data;

[0168] The desensitization coverage rate is calculated according to the number of fields that need to be desensitized and the number of fields that have been desensitized;

[0169] Verify the fields that need to be desensitized and the fields that have been desensitized to obtain the number of successfully desensitized fields;

[0170] The desensitization success rate is calculated according to the number of successful desensitization and the number of fields that need to be desensitized;

[0171] The first desensitization effect is calculated according to the desensitization coverage rate and the desensitization success rate.

[0172] Desensitization coverage = number of fields that need to be desensitized / number of fields that have been desensitized. Desensitization coverage is used to describe the coverage of the desensitization algorithm for the desensitized data. It can intuitively reflect whether the desensitization algorithm has the ability to process various typical data. The higher the desensitization coverage, the better the desensitization effect, and the lower the desensitization coverage, the worse the desensitization effect.

[0173] Desensitization success rate = number of successfully desensitized fields / number of fields that need to be desensitized. The desensitization success rate is used to characterize the record of the successful desensitization of the data in the data set by the desensitization algorithm. The higher the desensitization success rate, the better the desensitization effect, and the lower the desensitization success rate, the worse the desensitization effect.

[0174] Whether the field to be desensitized is successfully desensitized can be determined by verifying whether the length of the field to be desensitized is consistent with the length of the field that has been desensitized. If the lengths are consistent, it is determined that the desensitization is successful. If the lengths are inconsistent, it is determined that the desensitization fails. It is also possible to determine whether the field to be desensitized is successfully desensitized by verifying whether the field value to be desensitized is the same as the field value that has been desensitized. If the field values ​​are the same, it is determined that the desensitization is successful. If the field values ​​are different, it is determined that the desensitization fails.

[0175] The above optional implementation method calculates the desensitization coverage and desensitization success rate through the transaction data or transaction messages before and after desensitization, and combines the two dimensions of desensitization coverage and desensitization success rate to jointly determine the desensitization effect of each desensitization algorithm. The calculation accuracy of the desensitization effect is higher, which helps to determine the target desensitization algorithm, and then improve the desensitization quality of the target transaction data or target transaction messages.

[0176] The redundancy removal module 203 is used to synchronize the target transaction data and the target transaction message to the test environment, and remove redundancy from the target transaction message according to the target transaction data.

[0177] The test environment refers to the server hardware, software, network equipment, and historical data required to complete the test work.

[0178] The desensitized transaction data and transaction messages in the production environment are synchronized to the test environment for testing, so that the full range of transaction scenarios in the real production environment can be simulated in the test environment.

[0179] In order to improve the test efficiency of the target transaction data and the target transaction message in the test environment, it is necessary to remove redundancy from the target transaction data and the target transaction message.

[0180] In an optional implementation, the redundancy removal module 203 removes redundancy from the target transaction message according to the target transaction data, including:

[0181] Obtaining the target logical address and target message sequence number carried in the target transaction message;

[0182] Read the historical logical address carried in the historical transaction message and the historical message sequence number corresponding to the historical logical address;

[0183] Performing initial redundancy removal on the target transaction message according to the historical message sequence number and the target message sequence number;

[0184] Determining a transaction channel of the target transaction data, and classifying the target transaction data according to the transaction channel;

[0185] Redundancy removal is performed on each type of target transaction data, and secondary redundancy removal is performed on the target transaction message after the initial redundancy removal according to the target transaction data after the redundancy removal.

[0186] The logical address may be a logical address corresponding to a terminal of a business system corresponding to a production environment, and the message sequence number is a cyclically increasing sequence.

[0187] The historical message sequence numbers corresponding to the historical logical addresses carried by the historical transaction messages in the target partition are collected together to obtain a historical message sequence number set, and each target message sequence number in the target transaction message is matched one by one with each historical message sequence number in the historical message sequence number set. When the target message sequence number matches the historical message sequence number successfully, the target transaction message corresponding to the target message sequence number is retained, and when the target message sequence number fails to match the historical message sequence number, the target transaction message corresponding to the target message sequence number is deleted. By matching the historical message sequence number with the target message sequence number, the target transaction message corresponding to the target message sequence number that fails to match is deleted, thereby realizing the initial redundancy removal of the target transaction message.

[0188] In the test environment, the target transaction data can be classified according to the transaction channels to obtain multiple types of target transaction data. The transaction channels may include: UnionPay, ChinaNet, ChinaNet, MasterCard, MasterNet, Liantong, etc.

[0189] Some classes have a lot of target transaction data, while some classes have very little target transaction data. For the target transaction data of the class with a small number, all of them are retained. For the target transaction data of the class with a large number, some of the target transaction data can be selected for testing based on the target transaction data of the class with a small number. It is also possible to select some of the target transaction data from the target transaction data of the class with a large number for testing according to a preset ratio. After the target transaction data is de-redundant, it must be ensured that the target transaction message corresponds to the target transaction data one by one, so that a test case set can be generated based on the target transaction data and the target transaction message after de-redundancy.

[0190] The above optional implementation method performs initial redundancy removal on the target transaction message through the logical address and the message sequence number, which can avoid redundancy failure caused by different delays in transmitting transaction messages through different channels or changes in the message sequence caused by forwarding devices such as switches; classifying the target transaction data according to the transaction channel and performing redundancy removal can effectively ensure that the transaction data of each type of transaction channel can participate in the test; performing secondary redundancy removal on the target transaction message after the initial redundancy removal according to the target transaction data after the redundancy removal can effectively ensure that the target transaction data after the redundancy removal corresponds one-to-one with the target transaction message, which is convenient for the subsequent generation of a test case set; in addition, the number of target transaction data after redundancy removal is greatly reduced compared with the target transaction data before the redundancy removal, reducing the number of target transaction data tested is conducive to saving test time and improving test efficiency.

[0191] The generating module 204 is used to generate a test case set according to the target transaction message after redundancy is removed.

[0192] A test case set is a set of test inputs, execution conditions, and expected results compiled for a specific purpose in order to test a program path or verify whether a specific requirement is met.

[0193] In a test environment, you can manually call a self-developed tool to read the message in the database table and resend the message request to the authorized application.

[0194] In an optional implementation, the generating module 204 generates a test case set according to the target transaction message after redundancy removal, including:

[0195] Obtaining a message header label of the target transaction message after the redundancy is removed;

[0196] Obtaining a response message according to the message header label;

[0197] A test case set is generated according to the target transaction message after redundancy removal and the response message.

[0198] The corresponding response message can be obtained from the production environment according to the message header label, so that the target transaction message after redundancy removal and the response message are used as a test case set, and the test case set is stored in the test library for repeated use.

[0199] The execution module 205 is used to execute the test case set in a preset first version test environment to obtain a first test result, and execute the test case set in a preset second version test environment to obtain a second test result.

[0200] The preset first version test environment refers to an environment that is the same as the actual production environment, and the preset second version test environment refers to an environment that is higher than the actual production environment.

[0201] Use two different versions of the test environment to test the test case set at the same time, and replay it on the benchmark machine after obtaining the test results. Obtain the response message after the test for full comparison to show the test differences between different test environments.

[0202] The comparison module 206 is used to compare the first test result and the second test result, and output the comparison result.

[0203] The comparison result may include: the first test result is the same as the second test result, and the first test result is different from the second test result.

[0204] When the first test result is the same as the second test result, it indicates that the actual production environment can run stably.

[0205] When the first test result is different from the second test result, it indicates that a problem occurs in the actual production environment, for example, a requirement change occurs or a system code is changed.

[0206] In an optional implementation, the comparison module 206 is further configured to:

[0207] When the comparison result shows that the first test result is different from the second test result, identifying difference data between the first test result and the second test result;

[0208] A test report is generated according to the difference data.

[0209] The test report can contain data comparison and display results.

[0210] The above optional implementation mode generates a test report and displays the difference data between the first test result and the second test result intuitively, which can facilitate developers or testers to find the cause of the difference based on the difference data, so as to quickly propose corresponding solutions.

[0211] The embodiments of the present invention can significantly improve the comprehensiveness, accuracy, coverage and testing efficiency of data testing, thereby ensuring the stable operation of the production environment and reducing the possibility of production accidents.

[0212] Embodiment 3

[0213] This embodiment provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned big data comparison test method embodiment are implemented, such as Figure 1S11-S16 shown:

[0214] S11, determining a target partition in a production environment, and collecting transaction data and transaction messages in the target partition;

[0215] S12, performing desensitization processing on the transaction data based on the historical data of the target partition to obtain target transaction data, and performing desensitization processing on the transaction message to obtain a target transaction message;

[0216] S13, synchronizing the target transaction data and the target transaction message into the test environment, and removing redundancy from the target transaction message according to the target transaction data;

[0217] S14, generating a test case set according to the target transaction message after redundancy removal;

[0218] S15, executing the test case set in a preset first version test environment to obtain a first test result, and executing the test case set in a preset second version test environment to obtain a second test result;

[0219] S16, comparing the first test result and the second test result, and outputting the comparison result.

[0220] Alternatively, when the computer program is executed by a processor, the functions of each module / unit in the above-mentioned device embodiment are realized, for example Figure 2 Modules 201-206:

[0221] The collection module 201 is used to determine the target partition in the production environment and collect transaction data and transaction messages in the target partition;

[0222] The desensitizing module 202 is used to perform desensitization processing on the transaction data based on the historical data of the target partition to obtain target transaction data, and perform desensitization processing on the transaction message to obtain a target transaction message;

[0223] The redundancy removal module 203 is used to synchronize the target transaction data and the target transaction message to the test environment, and remove redundancy from the target transaction message according to the target transaction data;

[0224] The generating module 204 is used to generate a test case set according to the target transaction message after redundancy removal;

[0225] The execution module 205 is used to execute the test case set in a preset first version of the test environment to obtain a first test result, and execute the test case set in a preset second version of the test environment to obtain a second test result;

[0226] The comparison module 206 is used to compare the first test result and the second test result, and output the comparison result.

[0227] Embodiment 4

[0228] See also Figure 3 FIG. 2 is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present invention. In a preferred embodiment of the present invention, the electronic device 3 includes a memory 31 , at least one processor 32 , at least one communication bus 33 and a transceiver 34 .

[0229] Those skilled in the art should understand that Figure 3 The structure of the electronic device shown does not constitute a limitation of the embodiments of the present invention, and may be a bus structure or a star structure. The electronic device 3 may also include more or less other hardware or software than shown in the figure, or a different component arrangement.

[0230] In some embodiments, the electronic device 3 is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application-specific integrated circuits, programmable gate arrays, digital processors, and embedded devices. The electronic device 3 may also include client devices, which include but are not limited to any electronic product that can interact with a client through a keyboard, mouse, remote control, touchpad, or voice-controlled device, such as a personal computer, tablet computer, smart phone, digital camera, etc.

[0231] It should be noted that the electronic device 3 is only an example, and other existing or future electronic products that are suitable for the present invention should also be included in the protection scope of the present invention and included here by reference.

[0232] In some embodiments, the memory 31 stores a computer program, and when the computer program is executed by the at least one processor 32, all or part of the steps in the big data comparison test method are implemented. The memory 31 includes a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0233] Furthermore, the computer-readable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the blockchain node, etc.

[0234] The blockchain referred to in this invention is a new application mode of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0235] In some embodiments, the at least one processor 32 is the control core (ControlUnit) of the electronic device 3, and uses various interfaces and lines to connect the various components of the entire electronic device 3, and executes various functions and processes data of the electronic device 3 by running or executing the program or module stored in the memory 31, and calling the data stored in the memory 31. For example, when the at least one processor 32 executes the computer program stored in the memory, it implements all or part of the steps of the big data comparison test method described in the embodiment of the present invention; or implements all or part of the functions of the big data comparison test device. The at least one processor 32 can be composed of an integrated circuit, for example, it can be composed of a single packaged integrated circuit, or it can be composed of multiple integrated circuits with the same function or different functions, including one or more central processing units (CPU), microprocessors, digital processing chips, graphics processors, and various control chips.

[0236] In some embodiments, the at least one communication bus 33 is configured to implement connection and communication between the memory 31 and the at least one processor 32, etc.

[0237] Although not shown, the electronic device 3 may also include a power source (such as a battery) for supplying power to each component. Preferably, the power source may be logically connected to the at least one processor 32 through a power management device, so that the power management device can manage charging, discharging, and power consumption management. The power source may also include one or more DC or AC power sources, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device 3 may also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.

[0238] The above-mentioned integrated unit implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, electronic device, or network device, etc.) or a processor to execute a part of the method described in each embodiment of the present invention.

[0239] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0240] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, and may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0241] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0242] It is obvious to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential features of the present invention. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-restrictive, and the scope of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any figure mark in the claims should not be regarded as limiting the claims involved. In addition, it is obvious that the word "including" does not exclude other units or, and the singular does not exclude the plural. Multiple units or devices stated in the specification may also be implemented by one unit or device through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any particular order.

[0243] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A big data comparison test method, characterized in that: The method comprises: Determine a target partition in a production environment, and collect transaction data and transaction messages in the target partition, including: obtaining acquisition control timing information of a data acquisition configuration process; matching each timing node in the acquisition control timing information with a preset timing allocation program to obtain process allocation control information; and collecting transaction data and transaction messages in the target partition according to the process allocation control information; Based on the historical data of the target partition, the transaction data is desensitized to obtain the target transaction data, and the transaction message is desensitized to obtain the target transaction message; wherein the historical data of the target partition includes historical transaction data and historical transaction messages; The target transaction data and the target transaction message are synchronized to the test environment, and the target transaction message is de-redundant according to the target transaction data, including: obtaining the target logical address and the target message sequence number carried in the target transaction message; reading the historical logical address carried in the historical transaction message and the historical message sequence number corresponding to the historical logical address; performing initial de-redundancy on the target transaction message according to the historical message sequence number and the target message sequence number; determining the transaction channel of the target transaction data, and classifying the target transaction data according to the transaction channel; de-redundant each type of target transaction data, and performing secondary de-redundancy on the target transaction message after initial de-redundancy according to the target transaction data after de-redundancy; Generate a test case set based on the target transaction message after redundancy removal; Execute the test case set in a preset first version test environment to obtain a first test result, and execute the test case set in a preset second version test environment to obtain a second test result; The first test result and the second test result are compared, and the comparison result is output.

2. The big data comparison test method according to claim 1, characterized in that: The acquisition control timing information of the data acquisition configuration process includes: Dividing the data acquisition configuration process into a plurality of unrelated configuration partition objects; Determine a partition service label of the configuration partition object; Extracting multiple groups of information from the data collection configuration process; The acquisition control timing information of the data acquisition configuration process is obtained according to each group of information extraction results and the partition service tag of each of the configured partition objects.

3. The big data comparison test method according to claim 1, characterized in that: The desensitizing the transaction data based on the historical data of the target partition to obtain the target transaction data, and desensitizing the transaction message to obtain the target transaction message includes: Desensitizing the historical transaction data using multiple first desensitizing algorithms to obtain first desensitized data corresponding to each of the first desensitizing algorithms, and desensitizing the historical transaction messages using multiple second desensitizing algorithms to obtain second desensitized data corresponding to each of the second desensitizing algorithms; Calculate a first desensitization effect of the first desensitization algorithm according to the historical transaction data and the first desensitization data, and calculate a second desensitization effect of the second desensitization algorithm according to the historical transaction message and the second desensitization data; Determine a first target desensitization algorithm from the multiple first desensitization algorithms according to the first desensitization effect, and use the first target desensitization algorithm to perform desensitization processing on the transaction data to obtain target transaction data; A second target desensitization algorithm is determined from the multiple second desensitization algorithms according to the second desensitization effect, and the transaction message is desensitized using the second target desensitization algorithm to obtain a target transaction message.

4. The big data comparison test method according to claim 3, characterized in that: The calculating, according to the historical transaction data and the first desensitized data, a first desensitization effect of the first desensitization algorithm corresponding to the first desensitization algorithm comprises: For any one of the first desensitizing algorithms, obtaining the number of fields that need to be desensitized in the historical transaction data and obtaining the number of fields that have been desensitized in the first desensitized data; The desensitization coverage rate is calculated according to the number of fields that need to be desensitized and the number of fields that have been desensitized; Verify the fields that need to be desensitized and the fields that have been desensitized to obtain the number of successfully desensitized fields; The desensitization success rate is calculated according to the number of successful desensitization and the number of fields that need to be desensitized; The first desensitization effect is calculated according to the desensitization coverage rate and the desensitization success rate.

5. The big data comparison test method according to any one of claims 1 to 4, characterized in that: The generating of the test case set according to the target transaction message after redundancy removal comprises: Obtaining a message header label of the target transaction message after the redundancy is removed; Obtaining a response message according to the message header label; A test case set is generated according to the target transaction message after redundancy removal and the response message.

6. A big data comparison test device, comprising a module for implementing the method according to any one of claims 1 to 5, characterized in that: The device comprises: A collection module, used to determine a target partition in a production environment and collect transaction data and transaction messages in the target partition; A desensitizing module, configured to perform desensitizing processing on the transaction data based on the historical data of the target partition to obtain target transaction data, and perform desensitizing processing on the transaction message to obtain a target transaction message; A redundancy removal module, used to synchronize the target transaction data and the target transaction message to the test environment, and remove redundancy from the target transaction message according to the target transaction data; A generation module, used to generate a test case set according to the target transaction message after redundancy removal; An execution module, configured to execute the test case set in a preset first version of the test environment to obtain a first test result, and execute the test case set in a preset second version of the test environment to obtain a second test result; The comparison module is used to compare the first test result and the second test result, and output the comparison result.

7. An electronic device, characterized in that: The electronic device includes a processor and a memory, and the processor is used to implement the big data comparison test method as described in any one of claims 1 to 5 when executing the computer program stored in the memory.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the big data comparison test method as described in any one of claims 1 to 5 is implemented.

Images