Data processing method and device, equipment, medium and program product

Abstract

The invention provides a data processing method which can be applied to the technical field of big data. The data processing method comprises the following steps: in response to execution of at least one transaction request in a production environment, obtaining first transaction data generated by execution of each transaction request, and when execution of each transaction request is completed, synchronously obtaining the first transaction data corresponding to the transaction request; desensitizing the first transaction data, deleting or converting sensitive information in the first transaction data, and obtaining at least one piece of second transaction data; storing the at least one piece of second transaction data in a first memory space, wherein the first memory space is a storage space independent of the production environment and the test environment; determining at least one piece of second transaction data from the at least one piece of second transaction data as target transaction data; and transmitting the target transaction data to a test environment so as to perform a test based on the target transaction data in the test environment. The invention further provides a data processing device, equipment, a storage medium and a program product.

Description

Technical Field

[0001] The present disclosure relates to the field of big data, and more specifically to a data processing method, apparatus, device, medium and program product. Background Art

[0002] In banking systems, data synchronization and problem reproduction between the production environment and the test environment is a complex and critical task. In the traditional reproduction method, developers usually go to the data lake to check and analyze data when encountering production problems to find the root cause of the problem. Subsequently, testers will simulate the production environment configuration parameters in the test environment according to the operations of business personnel, and reproduce the data to verify the solution. However, this solution requires a lot of manpower and time to perform system data generation, especially when it involves cross-application information preparation, which often requires the cooperation of upstream and downstream systems, seriously affecting the efficiency of problem follow-up and the timeliness of solution verification.

[0003] Based on this, a solution has been proposed to directly copy the transaction data in the production environment to the test environment and store it in the test environment. When the transaction data of the problematic customer appears, the tester filters the corresponding data from the data stored in the test environment to generate test data. Although this method solves the problem of complex system data generation, it also leads to excessive data volume in the test environment, increased storage costs, and affects the query and test efficiency in the test environment, thereby affecting the stability of the entire test environment. Summary of the invention

[0004] In view of the above problems, the present disclosure provides a data processing method, apparatus, device, medium and program product.

[0005] According to a first aspect of the present disclosure, a data processing method is provided, comprising: in response to executing at least one transaction request in a production environment, obtaining first transaction data generated by executing each transaction request, wherein when each transaction request is executed, synchronously obtaining the first transaction data corresponding to the transaction request; desensitizing the first transaction data, deleting or converting sensitive information in the first transaction data, and obtaining at least one second transaction data; storing the at least one second transaction data in a first memory space, the first memory space being a storage space independent of a production environment and a test environment; determining at least one second transaction data as target transaction data from the at least one second transaction data; and transmitting the target transaction data to a test environment, so as to perform testing based on the target transaction data in the test environment.

[0006] According to an embodiment of the present disclosure, determining at least one second transaction data as target transaction data from at least one second transaction data includes: determining second transaction data with abnormal transactions in at least one transaction request as target transaction data; and / or determining second transaction data with normal transactions in at least one transaction request as target transaction data.

[0007] According to an embodiment of the present disclosure, determining at least one second transaction data as target transaction data from at least one second transaction data also includes: obtaining first information of each second transaction data in the at least one second transaction data, the first information being used to mark an abnormal state of the second transaction data; and determining the target transaction data from the at least one second transaction data based on the first information.

[0008] According to an embodiment of the present disclosure, it is characterized in that it also includes: before transmitting the target transaction data to the test environment, obtaining second information of the target transaction data, the second information is used to characterize the business type of the target transaction data; based on the business type of the target transaction data, obtaining a data configuration template of the target transaction data from at least one data configuration template pre-stored in the first memory space, the data configuration template is used for batch screening and acquisition of data; based on the data configuration template of the target transaction data, screening and acquiring the target transaction data from at least one second transaction data.

[0009] According to an embodiment of the present disclosure, it is characterized in that it also includes: before testing based on the target transaction data in the test environment, in response to an abnormality in the target transaction data, deleting the target transaction data; updating the target transaction data, and transmitting the updated target transaction data to the test environment.

[0010] According to an embodiment of the present disclosure, it also includes: in response to executing at least one transaction request in a production environment, obtaining a first configuration parameter, the first configuration parameter being used to represent a configuration parameter of the production environment corresponding to the first transaction data generated by executing each transaction request; and storing the first configuration parameter in a first memory space.

[0011] According to an embodiment of the present disclosure, it also includes: acquiring a second configuration parameter, the second configuration parameter is used to represent a configuration parameter in a test environment; in response to a match between the first configuration parameter and the second configuration parameter corresponding to the target transaction data, determining difference information; and storing the difference information in a first memory space.

[0012] According to an embodiment of the present disclosure, the method further includes: updating the second configuration parameter according to the difference information.

[0013] A second aspect of the present disclosure provides a transaction data generating device, the device comprising: an acquisition module, for acquiring first transaction data generated by executing each transaction request in response to executing at least one transaction request in a production environment, wherein when each transaction request is executed, the first transaction data corresponding to the transaction request is synchronously acquired; a processing module, for desensitizing the first transaction data, deleting or converting sensitive information in the first transaction data, and obtaining at least one second transaction data; a storage module, for storing the at least one second transaction data in a first memory space, the first memory space being a storage space independent of a production environment and a test environment; and a generation module, for determining at least one second transaction data as target transaction data from the at least one second transaction data; and transmitting the target transaction data to a test environment, so as to perform testing based on the target data in the test environment.

[0014] A third aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0015] The fourth aspect of the present disclosure further provides a computer-readable storage medium having a computer program or instructions stored thereon, which implements the steps of the above method when the above computer program or instructions are executed by a processor.

[0016] The fifth aspect of the present disclosure further provides a computer program product, including a computer program or instructions, which implement the steps of the above method when the above computer program or instructions are executed by a processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The above contents and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0018] Figure 1 The application scenario diagram of the data processing method, apparatus, device, medium and program product according to the embodiments of the present disclosure is schematically shown;

[0019] Figure 2 The flowchart of the data processing method according to the first embodiment of the present disclosure is schematically shown;

[0020] Figure 3 A flowchart schematically shows a data processing method according to the second embodiment of the present disclosure;

[0021] Figure 4 A flowchart schematically shows a data processing method according to a third embodiment of the present disclosure;

[0022] Figure 5A flowchart schematically shows a data processing method according to a fourth embodiment of the present disclosure;

[0023] Figure 6 A flowchart schematically shows a data processing method according to a fifth embodiment of the present disclosure;

[0024] Figure 7 A flowchart schematically shows a data processing method according to a sixth embodiment of the present disclosure;

[0025] Figure 8 A structural block diagram schematically shows a data processing device according to an embodiment of the present disclosure; and

[0026] Fig. 9 A block diagram of an electronic device suitable for implementing a data processing method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0027] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0028] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0029] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0030] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0031] In the technical solution of the present disclosure, the user information (including but not limited to user personal information, user image information, user device information, such as location information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0032] In the scenario of using personal information for automated decision-making, the methods, devices, and systems provided by the embodiments of the present disclosure provide users with corresponding operation portals for users to choose to agree or reject the automated decision-making results; if the user chooses to reject, the expert decision-making process will be entered. The expression "automated decision-making" here refers to the activity of automatically analyzing and evaluating a person's behavioral habits, interests and hobbies, or economic, health, credit status, etc. through computer programs, and making decisions. The expression "expert decision-making" here refers to the activity of making decisions by people who specialize in a certain field, have specialized experience, knowledge and skills, and have reached a certain level of professionalism.

[0033] Figure 1 The application scenario diagram of the data processing method according to the embodiment of the present disclosure is schematically shown.

[0034] like Figure 1 As shown, the application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a network 103, and a server 104. The network 103 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, and the server 104. The network 103 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0035] The user can use the first terminal device 101 and the second terminal device 102 to interact with the server 104 through the network 103 to receive or send messages, etc.

[0036] The first terminal device 101 and the second terminal device 102 may be various electronic devices having a display screen and supporting banking transactions, including but not limited to a smart phone, a tablet computer, a laptop computer, a desktop computer, and the like.

[0037] The server 104 may be a server that provides various services, such as a background management server (only an example) that provides support for obtaining transaction data generated by a user using the first terminal device 101 and the second terminal device 102 for business transactions. The production environment generates transaction data in response to a transaction request from a terminal device and stores it in the production environment. At the same time, the background management server synchronously obtains the generated transaction data and transmits the target transaction data extracted from the transaction data to the test environment, wherein the production environment and the test environment are two other servers different from the background management server.

[0038] It should be noted that the data processing method provided in the embodiment of the present disclosure can generally be executed by the server 104. Accordingly, the data processing device provided in the embodiment of the present disclosure can generally be set in the server 104. The data processing method provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 104 and can communicate with the first terminal device 101, the second terminal device 102 and / or the server 104. Accordingly, the data processing device provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 104 and can communicate with the first terminal device 101, the second terminal device 102 and / or the server 104.

[0039] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.

[0040] The following will be based on Figure 1 The scene described by Figure 2~Figure 7 The data processing method of the disclosed embodiment is described in detail.

[0041] Figure 2 The flowchart of the data processing method according to the first embodiment of the present disclosure is schematically shown.

[0042] like Figure 2 As shown, the data processing of this embodiment includes operations S210 to S250, and the transaction processing method can be executed by the server 104.

[0043] In operation S210, in response to executing at least one transaction request in the production environment, first transaction data generated by executing each transaction request is acquired, wherein when each transaction request is executed, the first transaction data corresponding to the transaction request is synchronously acquired;

[0044] In operation S220, the first transaction data is desensitized, sensitive information in the first transaction data is deleted or converted, and at least one second transaction data is obtained;

[0045] In operation S230, at least one second transaction data is stored in a first memory space, where the first memory space is a storage space independent of a production environment and a test environment;

[0046] In operation S240, at least one second transaction data is determined as target transaction data from the at least one second transaction data;

[0047] In operation S250 , the target transaction data is transmitted to a test environment so as to perform a test based on the target transaction data in the test environment.

[0048] In the embodiments of the present disclosure, the production environment may be the actual operating environment of a banking system, processing real user transactions and business operations.

[0049] A test environment can be an environment isolated from the production environment and is used to test new features, fix issues, verify solutions, etc.

[0050] The first memory space may be a storage space independent of a storage area physically or logically isolated from the production environment and the test environment, and may be located in the server 104 or may be independent of the server 104 .

[0051] The first transaction message may be original, unprocessed transaction data, data captured directly from a production environment, and may contain complete transaction information and sensitive information.

[0052] Sensitive information can be protected personal information or business secrets, such as customer name, ID number, bank card number, etc.

[0053] For example, multiple transaction requests such as transfers, payments, loans, deposits, etc. are generated in the production environment. Among them, transfers can include transfer transactions, inter-bank transfers and international transfers; payments can include credit card payments, debit card payments and e-wallet payments; loans can include loan applications and loan repayments; deposits can include time deposits and demand deposits.

[0054] When each transaction request is completed, the full-journey transaction data generated by each transaction request is synchronously obtained as the first transaction data. In the banking system, the full-journey transaction data refers to all information involved from the initiation of the transaction request to the completion of the transaction, including all intermediate steps and associated data, which not only covers the basic information of the transaction, but also includes all activity records, state changes and any associated business processes related to the transaction, such as host databases, collaborative tables and logs.

[0055] Each acquired first transaction data is desensitized to protect the user's sensitive information and generate the corresponding second transaction data. For example, the customer ID and payment amount can be converted through a hash algorithm, and other sensitive fields can be shielded or randomized. Sensitive information can also be replaced with predefined false data to maintain the authenticity and consistency of the data.

[0056] The second transaction data is stored in an independent first memory space, isolated from the production environment and the test environment, to ensure data security and isolation. The target transaction data is determined from the second data. The target transaction data at this time can be a single second transaction data or a batch of second transaction data. The determined target transaction data is then transferred to the test environment. Testers can use these target data in the test environment for operations such as function verification, performance testing, and troubleshooting.

[0057] According to the embodiments of the present disclosure, this embodiment avoids direct data exposure risks by synchronously acquiring the generated transaction data in real time, performing desensitizing processing and storing it in the first memory space, isolating it from the production environment and the test environment; reducing the data preparation time from the production environment to the test environment, improving the speed of problem reproduction and solution verification, and reducing labor costs and time consumption; and only transmitting necessary target transaction data, reducing the amount of data in the test environment, reducing the consumption of storage and computing resources and operating costs, while improving the response speed of queries and operations, and ensuring the overall performance of the system.

[0058] Based on the above embodiment, determining at least one second transaction data as target transaction data from at least one second transaction data may include:

[0059] determining second transaction data with transaction abnormality in at least one transaction request as target transaction data; and / or,

[0060] The second transaction data with normal transaction in at least one transaction request is determined as the target transaction data.

[0061] In the embodiments of the present disclosure, for example, data with abnormal transactions can be selected as target transaction data, and resources can be concentrated to conduct in-depth analysis and resolution of the problem. For example, testers can directly debug and repair abnormal situations, shortening the time cycle for problem discovery and resolution; and by analyzing abnormal data, potential risk points in the system can be identified, so that measures can be taken to improve the stability and reliability of the system.

[0062] For example, you can select normal transaction data as target transaction data, and conduct comprehensive functional verification and performance evaluation in the test environment. For example, by reproducing the normal transaction process, you can verify whether the performance of the system under normal operation is consistent with the production environment, ensuring the functional integrity of the system; using normal transaction data for stress testing can help identify the performance bottlenecks of the system under high concurrency and large data volume, and optimize system performance; and by analyzing the user's behavior path and operation mode, you can optimize the interface design and interaction logic and improve the user experience.

[0063] For example, you can choose to combine abnormal transaction data with normal transaction data. Considering both abnormal and normal data at the same time can more comprehensively test all aspects of the system and ensure the overall quality of the system.

[0064] According to the embodiments of the present disclosure, by providing a method for flexibly selecting abnormal transaction data and normal transaction data as target transaction data, the targeted troubleshooting is significantly enhanced, the functions and performance of the system are comprehensively verified, different testing requirements can be flexibly responded to, and the testing efficiency and accuracy are improved.

[0065] Figure 3 The flowchart of the data processing method according to the second embodiment of the present disclosure is schematically shown.

[0066] like Figure 3 As shown, the data processing of this embodiment includes operations S310 to S320, and the transaction processing method can be executed by the server 104.

[0067] Based on the above embodiment, determining at least one second transaction data as target transaction data from at least one second transaction data further includes:

[0068] In operation S310, first information of each second transaction data in at least one second transaction data is obtained, where the first information is used to mark an abnormal state of the second transaction data;

[0069] In operation S320, target transaction data is determined from at least one second transaction data based on the first information.

[0070] In the embodiment of the present disclosure, the first information of the second transaction data is obtained to identify and distinguish normal transaction data from abnormal transaction data. For example, when the payment transaction request is completed, the generated second transaction data will carry the first information marking the abnormal state.

[0071] Whether to include only abnormal transactions, only normal transactions, or both types of transaction data may be determined based on the determined first information.

[0072] According to the embodiments of the present disclosure, by clearly marking the abnormal status of each transaction, abnormal transactions that need to be focused on can be quickly screened out, and resources can be concentrated for analysis and repair; at the same time, it supports the selection of abnormal or normal transaction data according to different testing requirements, providing a more flexible testing strategy.

[0073] Figure 4 The flowchart of the data processing method according to the third embodiment of the present disclosure is schematically shown.

[0074] like Figure 4 As shown, the data processing of this embodiment includes operations S410 to S430, and the transaction processing method can be executed by the server 104.

[0075] Based on the above embodiment, it also includes:

[0076] In operation S410, before transmitting the target transaction data to the test environment, second information of the target transaction data is obtained, where the second information is used to characterize the business type of the target transaction data;

[0077] In operation S420, based on the business type of the target transaction data, a data configuration template of the target transaction data is obtained from at least one data configuration template pre-stored in the first memory space, the data configuration template being used for batch screening and obtaining data;

[0078] In operation S430, target transaction data is screened and acquired from at least one second transaction data based on the data configuration template of the target transaction data.

[0079] In an embodiment of the present disclosure, the business type of the transaction data may be transfer, payment, loan, etc. According to the determined business type of the target transaction data, a corresponding data configuration template is selected from a plurality of data configuration templates pre-stored in the first memory space. The data configuration template may, for example, include screening rules and acquisition logic for different business types.

[0080] For example, in the bank transaction system, there is abnormal payment transaction data, and this transaction data is determined to be the target transaction data, and the business type of this transaction data is determined to be payment. A payment data configuration template is selected from multiple pre-stored data configuration templates, and this payment data configuration template sets screening conditions, such as transaction ID, customer ID, payment method, amount, etc., as well as related data table names; by using the payment data configuration template, the target transaction data that meets the template conditions is screened out.

[0081] For example, if you need to obtain a batch of normal payment transaction data for testing, you can select a payment data configuration template from multiple pre-stored data configuration templates, set a filtering time period, and export batch data that meets the filtering conditions. Based on the filtered batch data, you can use the batch node scheduling tool to start the batch transaction processing node. The tool monitors the execution status of each node to ensure that all nodes run as planned and collect execution logs. After completion, check the data verification results according to the preset transaction rules to ensure the correctness and completeness of the batch transaction processing.

[0082] According to the embodiments of the present disclosure, target transaction data can be accurately screened and acquired from the second transaction data, cross-application data screening and acquisition can be supported, and the scale and distribution characteristics of production data can be simulated to perform batch transaction testing.

[0083] Figure 5 The flowchart of the data processing method according to the fourth embodiment of the present disclosure is schematically shown.

[0084] like Figure 5 As shown, the data processing of this embodiment includes operations S510 to S530, and the transaction processing method can be executed by the server 104.

[0085] Based on the above embodiment, it also includes:

[0086] In operation S510, before testing based on the target transaction data in the test environment, in response to an abnormality in the target transaction data, the target transaction data is deleted;

[0087] In operation S520, the target transaction data is updated, and the updated target transaction data is transmitted to the test environment.

[0088] In the embodiment of the present disclosure, before testing based on target transaction data in the test environment, if there are abnormalities in the target transaction data, the abnormal target transaction data are deleted, and the target transaction data are re-screened to obtain updates, and the updated data are transmitted to the test environment. Among them, the abnormality in this embodiment can be data missing, etc. In this implementation, by adopting the above method, it is possible to avoid affecting the accuracy and reliability of subsequent tests, and ensure the data quality in the test environment.

[0089] Figure 6 The flowchart of the data processing method according to the fifth embodiment of the present disclosure is schematically shown.

[0090] like Figure 6 As shown, the data processing of this embodiment includes operations S610 to S630, and the transaction processing method can be executed by the server 104.

[0091] According to an embodiment of the present disclosure, it also includes:

[0092] In operation S610, in response to executing at least one transaction request in the production environment, a first configuration parameter is obtained, the first configuration parameter being used to represent a configuration parameter of the production environment corresponding to first transaction data generated by executing each transaction request;

[0093] In operation S620, the first configuration parameter is stored in the first memory space.

[0094] According to an embodiment of the present disclosure, the configuration parameters may be various settings and conditions in a production environment to ensure that transaction requests can be correctly executed in a specific environment, including, for example, database connection strings, API endpoints, security policies, transaction limits, transaction rates, exchange rates, database server addresses, etc.

[0095] By obtaining the configuration parameters, i.e., the first configuration parameters, on which the transaction request depends each time the transaction request is executed in the production environment, and storing the obtained first configuration parameters in an independent first memory space, it is ensured that these parameters can be quickly accessed and used without affecting the normal operation of the production environment; at the same time, it is convenient to read the configuration parameters from the first memory space in subsequent steps for configuration synchronization of the test environment.

[0096] Based on the above embodiment, it also includes:

[0097] Before storing the first configuration parameter in the first memory space, a filtering condition may be set to obtain the filtered configuration parameter.

[0098] In the embodiments of the present disclosure, for example, screening rules can be defined, such as retaining only configuration parameters directly related to transaction processing; excluding configuration parameters that are insensitive or do not affect test results; retaining dynamically changing parameters such as exchange rates and rates; ignoring static parameters such as database connection strings, etc. The screened configuration parameters such as API endpoints, security policies, transaction limits, current exchange rates, and dynamic rates, etc. can be obtained through the above screening rules. The screened configuration parameters are then stored in the first memory space.

[0099] According to the embodiments of the present disclosure, unnecessary configuration item interference is reduced by screening and storing necessary configuration parameters.

[0100] Figure 7 The flowchart of the data processing method according to the sixth embodiment of the present disclosure is schematically shown.

[0101] like Figure 7 As shown, the data processing of this embodiment includes operations S710 to S730, and the transaction processing method can be executed by the server 104.

[0102] Based on the above embodiment, it also includes:

[0103] In operation S710, a second configuration parameter is obtained, where the second configuration parameter is used to represent a configuration parameter in a test environment;

[0104] In operation S720, in response to the first configuration parameter and the second configuration parameter corresponding to the target transaction data matching, difference information is determined;

[0105] In operation S730, the difference information is stored in the first memory space.

[0106] In an embodiment of the present disclosure, the first configuration parameter of the target transaction data in the production environment is compared item by item with the second configuration parameter in the test environment to determine the difference information between the two. Storing the determined configuration parameter difference information in the first memory space can ensure that the information is quickly accessed and used without affecting the normal operation of the production or test environment. At the same time, the generated difference information can be referenced to determine whether to optimize the configuration parameters of the test environment.

[0107] Based on the above embodiment, the method further includes: updating the second configuration parameter according to the difference information.

[0108] In the embodiment of the present disclosure, the difference information stored in the first memory space can be used to update the second configuration parameters in the test environment according to the needs in the actual test environment to ensure that it is consistent with the production environment. This step ensures that the test environment can accurately replicate the state of the production environment, improving the reliability and accuracy of the test results.

[0109] Based on the above data processing method, the present disclosure also provides a data processing device. Figure 8 The device is described in detail.

[0110] Figure 8 The structure block diagram of a data processing device according to an embodiment of the present disclosure is schematically shown.

[0111] like Figure 8 As shown, the data processing device 800 of this embodiment includes an acquisition module 810 , a processing module 820 , a storage module 830 and a generation module 840 .

[0112] The acquisition module 810 is used to, in response to executing at least one transaction request in the production environment, acquire the first transaction data generated by executing each transaction request, wherein when each transaction request is executed, the first transaction data corresponding to the transaction request is synchronously acquired. In one embodiment, the acquisition module 810 can be used to execute the operation S210 described above, which will not be described in detail here.

[0113] The processing module 820 is used to perform desensitization processing on the first transaction data, delete or convert sensitive information in the first transaction data, and obtain at least one second transaction data. In one embodiment, the processing module 820 can be used to perform the operation S220 described above, which will not be repeated here.

[0114] The storage module 830 is used to store at least one second transaction data in the first memory space, where the first memory space is a storage space independent of the production environment and the test environment. In one embodiment, the storage module 830 can be used to perform the operation S230 described above, which will not be described in detail here.

[0115] The generating module 840 is used to determine at least one second transaction data as target transaction data from at least one second transaction data; and transmit the target transaction data to the test environment to perform a test based on the target data in the test environment. In one embodiment, the generating module 840 can be used to perform the operations S240 and S250 described above, which will not be described in detail here.

[0116] According to an embodiment of the present disclosure, the above device further includes:

[0117] The update module 850 is used to delete the target transaction data in response to an abnormality in the target transaction data before testing based on the target transaction data in the test environment; update the target transaction data and transmit the updated target transaction data to the test environment. In one embodiment, the update module 850 can be used to perform the operations S510 and S520 described above, which will not be repeated here.

[0118] The difference module 860 is used to determine difference information in response to the first configuration parameter and the second configuration parameter corresponding to the target transaction data matching. In one embodiment, the difference module 860 can be used to perform the operation S720 described above, which will not be described in detail here.

[0119] According to an embodiment of the present disclosure, the acquisition module 810 can also be used to acquire the first information of each second transaction data in at least one second transaction data, the first information is used to mark the abnormal state of the second transaction data; before transmitting the target transaction data to the test environment, the second information of the target transaction data is acquired, the second information is used to characterize the business type of the target transaction data; in response to executing at least one transaction request in the production environment, the first configuration parameter is acquired, the first configuration parameter is used to represent the configuration parameter of the production environment corresponding to the first transaction data generated by executing each transaction request; and the second configuration parameter is acquired, the second configuration parameter is used to represent the configuration parameter in the test environment. In one embodiment, the acquisition module 810 can be used to perform the operations S310, S410, S610 and S710 described above, which will not be repeated here.

[0120] The storage module 830 may also be used to store the first configuration parameter in the first memory space; and store the difference information in the first memory space. In one embodiment, the storage module 830 may be used to perform the operations S620 and S730 described above, which will not be described in detail here.

[0121] The generation module 840 can also be used to determine the second transaction data with abnormal transactions in at least one transaction request as the target transaction data; and / or determine the second transaction data with normal transactions in at least one transaction request as the target transaction data; determine the target transaction data from at least one second transaction data based on the first information; obtain the data configuration template of the target transaction data from at least one data configuration template pre-stored in the first memory space based on the business type of the target transaction data, the data configuration template is used for batch screening and data acquisition; screen and acquire the target transaction data from at least one second transaction data based on the data configuration template of the target transaction data. In one embodiment, the generation module 840 can be used to perform the operations S320, S250, S420 and S430 described above, which will not be repeated here.

[0122] According to an embodiment of the present disclosure, any multiple modules among the acquisition module 810, the processing module 820, the storage module 830, the generation module 840, the difference module 840, and the update module 850 can be combined in one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the acquisition module 810, the processing module 820, the storage module 830, the generation module 840, the difference module 840, and the update module 850 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware or in a suitable combination of any of them. Alternatively, at least one of the acquisition module 810, the processing module 820, the storage module 830, the generation module 840, the difference module 840 and the update module 850 may be at least partially implemented as a computer program module, which may perform a corresponding function when executed.

[0123] Fig. 9 A block diagram of an electronic device suitable for implementing a data processing method according to an embodiment of the present disclosure is schematically shown.

[0124] like Fig. 9As shown, the electronic device 900 according to an embodiment of the present disclosure includes a processor 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage part 908 to a random access memory (RAM) 903. The processor 901 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a dedicated microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 901 may also include an onboard memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0125] In RAM 903, various programs and data required for the operation of electronic device 900 are stored. Processor 901, ROM 902 and RAM 903 are connected to each other via bus 904. Processor 901 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 902 and / or RAM 903. It should be noted that the program can also be stored in one or more memories other than ROM 902 and RAM 903. Processor 901 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0126] According to an embodiment of the present disclosure, the electronic device 900 may further include an input / output (I / O) interface 905, which is also connected to the bus 904. The electronic device 900 may further include one or more of the following components connected to the input / output (I / O) interface 905: an input portion 906 including a keyboard, a mouse, etc.; an output portion 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 908 including a hard disk, etc.; and a communication portion 909 including a network interface card such as a LAN card, a modem, etc. The communication portion 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the input / output (I / O) interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed, so that a computer program read therefrom is installed into the storage portion 908 as needed.

[0127] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0128] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present disclosure, a computer-readable storage medium may include the ROM 902 and / or RAM 903 described above and / or one or more memories other than ROM 902 and RAM 903.

[0129] The embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the data processing method provided by the embodiment of the present disclosure.

[0130] The above functions defined in the system / device of the embodiment of the present disclosure are performed when the computer program is executed by the processor 901. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0131] In one embodiment, the computer program may be based on a tangible storage medium such as an optical storage device, a magnetic storage device, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of a signal on a network medium, and downloaded and installed through the communication part 909, and / or installed from a removable medium 911. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0132] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 909, and / or installed from the removable medium 911. When the computer program is executed by the processor 901, the above functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0133] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0134] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0135] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations or combinations are not explicitly described in the present disclosure. In particular, without departing from the spirit and teachings of the present disclosure, the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0136] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A data processing method, characterized in that: The method comprises: In response to executing at least one transaction request in the production environment, obtaining first transaction data generated by executing each transaction request, wherein when each transaction request is executed, the first transaction data corresponding to the transaction request is synchronously obtained; Performing desensitization processing on the first transaction data, deleting or converting sensitive information in the first transaction data, and obtaining at least one second transaction data; Storing the at least one second transaction data in a first memory space, where the first memory space is a storage space independent of the production environment and the test environment; Determining at least one second transaction data as target transaction data from the at least one second transaction data; The target transaction data is transmitted to the test environment, so as to perform a test based on the target transaction data in the test environment.

2. The method according to claim 1, characterized in that The determining at least one second transaction data as target transaction data from the at least one second transaction data includes: determining the second transaction data with transaction abnormality in the at least one transaction request as the target transaction data; and / or, The second transaction data with normal transaction in the at least one transaction request is determined as the target transaction data.

3. The method according to claim 2, characterized in that The determining at least one second transaction data as target transaction data from the at least one second transaction data further includes: Acquire first information of each second transaction data in the at least one second transaction data, where the first information is used to mark an abnormal state of the second transaction data; The target transaction data is determined from the at least one second transaction data based on the first information.

4. The method according to claim 3, characterized in that: Also includes: Before transmitting the target transaction data to the test environment, obtaining second information of the target transaction data, where the second information is used to characterize a business type of the target transaction data; Based on the business type of the target transaction data, obtaining a data configuration template of the target transaction data from at least one data configuration template pre-stored in the first memory space, the data configuration template being used for batch screening and obtaining data; The target transaction data is filtered and acquired from the at least one second transaction data based on the data configuration template of the target transaction data.

5. The method according to claim 1, characterized in that Also includes: Before testing based on the target transaction data in the test environment, in response to an abnormality in the target transaction data, deleting the target transaction data; The target transaction data is updated, and the updated target transaction data is transmitted to the test environment.

6. The method according to claim 1, characterized in that Also includes: In response to executing at least one transaction request in the production environment, obtaining a first configuration parameter, the first configuration parameter being used to represent a configuration parameter of the production environment corresponding to first transaction data generated by executing each transaction request; The first configuration parameter is stored in the first memory space.

7. The method according to claim 6, characterized in that Also includes: Acquire a second configuration parameter, where the second configuration parameter is used to represent a configuration parameter in the test environment; In response to the first configuration parameter and the second configuration parameter corresponding to the target transaction data matching, determining difference information; The difference information is stored in the first memory space.

8. The method according to claim 7, characterized in that Also includes: The second configuration parameter is updated according to the difference information.

9. A transaction data generating device, characterized in that: The device comprises: an acquisition module, configured to, in response to executing at least one transaction request in the production environment, acquire first transaction data generated by executing each transaction request, wherein when each transaction request is executed, the first transaction data corresponding to the transaction request is synchronously acquired; a processing module, configured to perform desensitization processing on the first transaction data, delete or convert sensitive information in the first transaction data, and obtain at least one second transaction data; a storage module, configured to store the at least one second transaction data in a first memory space, wherein the first memory space is a storage space independent of the production environment and the test environment; and A generating module is used to determine at least one second transaction data from the at least one second transaction data as target transaction data; and transmit the target transaction data to the test environment to perform testing based on the target data in the test environment.

10. An electronic device comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

12. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images