Test data distribution method and device, equipment, storage medium and program product

Abstract

The invention discloses a test data distribution method and device, equipment, a storage medium and a program product, and relates to the technical field of data processing.The test data distribution method comprises the steps that test data configuration information of a user is received; determining a target data structure, a data generation rule, a target distribution mode and target storage middleware according to the test data configuration information and a preset metadata warehouse; according to the data generation rule, constructing data conforming to the target data structure to obtain test data; and based on the target distribution mode, distributing the test data to the target storage middleware. According to the scheme, the test data can be automatically distributed to the multiple storage middleware after being generated at a time, repeated data generation of the multi-source storage middleware is effectively avoided, the accuracy of the test data is ensured, and the test efficiency is effectively improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a test data distribution method, device, equipment, storage medium and program product. Background Art

[0002] Existing test data generation mechanisms generally fail to support complex synchronization tasks across different database types. Therefore, when dealing with diverse data storage middleware, testers often need to build test data for each database or storage system separately, which not only consumes a lot of time and human resources, but also easily leads to data inconsistency errors. At the same time, the test data generation process is complex and cumbersome, involving tedious script writing and data filling, which significantly increases the preparation and maintenance costs of the test.

[0003] From the above, it can be seen that how to efficiently and accurately distribute test data among various data storage middleware and reduce the complexity of test data generation is an urgent problem to be solved.

[0004] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0005] The main purpose of the present application is to provide a test data distribution method, apparatus, device, storage medium and program product, aiming to solve the technical problem of how to efficiently and accurately distribute test data among various data storage middleware and reduce the complexity of test data generation.

[0006] To achieve the above objectives, the present application proposes a test data distribution method, the method comprising:

[0007] Receive test data configuration information from the user;

[0008] Determine the target data structure, data generation rules, target distribution mode and target storage middleware according to the test data configuration information and the preset metadata warehouse;

[0009] According to the data generation rule, construct data that conforms to the target data structure to obtain test data;

[0010] Based on the target distribution mode, the test data is distributed to the target storage middleware.

[0011] In one embodiment, before the step of determining the target data structure, data generation rules, target distribution mode and target storage middleware according to the test data configuration information and the preset metadata repository, the following steps are included:

[0012] Identify the target basic data source and extract some data structure information;

[0013] The metadata warehouse is constructed according to the plurality of data structure information.

[0014] In one embodiment, the step of constructing data conforming to the target data structure according to the data generation rule to obtain test data includes:

[0015] Using a preset script and according to the data generation rule, construct data that conforms to the target data structure to obtain first constructed data;

[0016] Performing a data quality check on the first structured data based on a preset data quality check rule;

[0017] When the data quality check fails, the user is fed back to reconfigure the test data;

[0018] When the data quality check is qualified, the first constructed data is used as test data.

[0019] In one embodiment, the target storage middleware may be configured as a plurality of target storage middlewares, and the step of distributing the test data to the target storage middlewares based on the target distribution mode includes:

[0020] According to the target distribution mode, confirming the data structure to be updated in the target storage middleware;

[0021] For each target storage middleware, the test data is adapted and converted accordingly to obtain a number of adapted test data;

[0022] The plurality of adapted test data are synchronously and parallelly distributed to a plurality of target storage middlewares by using a pre-created thread pool, and the data structure to be updated is synchronously updated.

[0023] In one embodiment, before the step of distributing the test data to the target storage middleware based on the target distribution mode, the method further includes:

[0024] Determine data validation rules and conflict resolution strategies based on the test data configuration information;

[0025] Based on the data verification rules, verify the test data to confirm whether there is a data conflict;

[0026] When the stored data conflicts, the test data is corrected based on the conflict resolution strategy.

[0027] In one embodiment, after the step of distributing the test data to the target storage middleware based on the target distribution mode, the method further includes:

[0028] Performing a back check on the target storage middleware to confirm whether the test data is distributed successfully;

[0029] When the test data distribution fails, the data is resent or reissued according to a preset compensation mechanism.

[0030] In addition, to achieve the above-mentioned purpose, the present application also proposes a test data distribution device, the test data distribution device comprising:

[0031] A receiving module, used for receiving the test data configuration information of the user;

[0032] A parsing module, used to determine the target data structure, data generation rules, target distribution mode and target storage middleware according to the test data configuration information and the preset metadata warehouse;

[0033] A data construction module, used to construct data conforming to the target data structure according to the data generation rule to obtain test data;

[0034] A data distribution module is used to distribute the test data to the target storage middleware based on the target distribution mode.

[0035] In addition, to achieve the above objectives, the present application also proposes a test data distribution device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the test data distribution method described above.

[0036] In addition, to achieve the above objectives, the present application also proposes a storage medium, which is a computer-readable storage medium, and stores a computer program on the storage medium. When the computer program is executed by a processor, the steps of the test data distribution method described above are implemented.

[0037] In addition, to achieve the above-mentioned purpose, the present application also proposes a computer program product, wherein the computer program product includes a computer program, and when the computer program is executed by a processor, the steps of the test data distribution method described above are implemented.

[0038] One or more technical solutions proposed in this application have at least the following technical effects:

[0039] The test data distribution method, apparatus, device, storage medium and program product proposed in the embodiments of the present application specifically receive the user's test data configuration information; determine the target data structure, data generation rules, target distribution mode and target storage middleware according to the test data configuration information and a preset metadata repository; construct data that conforms to the target data structure according to the data generation rules to obtain test data; and distribute the test data to the target storage middleware based on the target distribution mode.

[0040] The present application receives the user's test data configuration information, and parses the test data configuration information and the preset metadata warehouse to obtain relevant information such as the target data structure, data generation rules, target distribution mode and target storage middleware; then, according to the data generation rules, constructs data that conforms to the target data structure to obtain test data, and distributes the test data to the target storage middleware based on the target distribution mode. This can achieve accurate generation of test data based on user test requirements, and automatically distribute test data to multiple storage middleware after it is generated once, effectively avoiding duplicate data generation in multi-source storage middleware, improving the efficiency of test data distribution between diverse data storage middleware, and further improving test efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0042] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0043] Figure 1 A flowchart of the first embodiment of the test data distribution method of the present application is provided;

[0044] Figure 2 A flowchart of the second embodiment of the test data distribution method of the present application is provided;

[0045] Figure 3 A flowchart of the third embodiment of the test data distribution method of the present application is provided;

[0046] Figure 4 A flowchart of the fourth embodiment of the test data distribution method of the present application is provided;

[0047] Figure 5 This is an example flow chart of a test data distribution method involved in an embodiment of the present application;

[0048] Figure 6 This is a schematic diagram of the module structure of the test data distribution device according to an embodiment of the present application;

[0049] Figure 7 This is a schematic diagram of the device structure of the hardware operating environment involved in the test data distribution method in the embodiment of the present application.

[0050] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0051] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0052] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0053] The main solution of the embodiment of the present application is: receiving the user's test data configuration information; determining the target data structure, data generation rules, target distribution mode and target storage middleware according to the test data configuration information and the preset metadata repository; constructing data that conforms to the target data structure according to the data generation rules to obtain test data; and distributing the test data to the target storage middleware based on the target distribution mode.

[0054] Faced with the rich diversity of business data types, the requirements for business continuity, and the compatibility support requirements for cross-technology stacks, enterprises often adopt a multi-source middleware storage strategy. This strategy can not only cope with complex and changing business needs, but also ensure the excellent performance, reliable stability, strict security and maximum cost-effectiveness of the data storage system.

[0055] However, the design of test cases for databases usually involves the generation of test data. Existing test data generation mechanisms generally fail to support complex synchronization tasks across different database types. Therefore, when dealing with diverse data storage middleware, testers are required to manually generate test data for a single data storage middleware. The test data generation process is complex and cumbersome, involving tedious script writing and data filling, which increases the complexity of test data generation and makes it impossible to guarantee the accuracy of test data, thereby reducing the efficiency of database-related tests.

[0056] From the above, it can be seen that how to efficiently and accurately distribute test data among various data storage middleware and reduce the complexity of test data generation is an urgent problem to be solved.

[0057] The present application provides a solution, which receives test data configuration information from a user, and parses the test data configuration information and a preset metadata warehouse to obtain relevant information such as a target data structure, data generation rules, a target distribution mode, and a target storage middleware; then, according to the data generation rules, data that conforms to the target data structure is constructed to obtain test data, and based on the target distribution mode, the test data is distributed to the target storage middleware. This can achieve accurate generation of test data based on user test requirements, and automatic distribution of test data to multiple storage middleware after it is generated once, effectively avoiding duplicate data generation in multi-source storage middleware, improving the efficiency of test data distribution between diverse data storage middleware, and further improving test efficiency.

[0058] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or a test data distribution device capable of realizing the above functions, etc. The following takes the test data distribution device as an example to illustrate this embodiment and the following embodiments.

[0059] Based on this, the present application embodiment provides a test data distribution method, referring to Figure 1 , Figure 1 This is a flowchart of the first embodiment of the test data distribution method of the present application.

[0060] In this embodiment, the test data distribution method includes steps S110 to S140:

[0061] Step S110, receiving the user's test data configuration information;

[0062] Specifically, the test data distribution device first needs to receive the user's test data generation requirements. Specifically, it can receive the user's test data configuration information through a visual page through multiple form components. In addition, it can also receive the user's test data configuration information through a graphical user interface (Graphical User Interface, GUI) or command line tool to understand the specific generation rules of test data, the importance of test data generation, the target data structure identification and other information.

[0063] Among them, the user's test data configuration information often includes information such as the source of original data, test data type and format, test data generation scope, the importance of test data generation tasks, and designated distributed data storage middleware. The test data generation scope refers to the complete range of time periods, geographical areas, numerical intervals, or other dimensions that the test data set needs to cover; data storage middleware refers to an intermediate layer software or service that can be used to store and manage data from different data sources (including databases, file systems, message queues, cloud services, IoT devices, etc.), including but not limited to relational databases such as MySQL, cache services such as Redis, distributed storage services such as HBASE, and message queue storage services such as Kafka.

[0064] Step S120, determining a target data structure, data generation rules, target distribution mode and target storage middleware according to the test data configuration information and the preset metadata repository;

[0065] It should be noted that data structure refers to a format for data organization, management and storage, and is a collection of data elements that have one or more specific relationships with each other. Data structure defines the logical and physical relationships between data elements. The preset metadata warehouse refers to a data storage platform pre-built by relevant personnel based on the needs of actual test data generation, which is used to store a variety of data structure information.

[0066] Specifically, the test data distribution device parses the name or identification information of the data structure required for the test data generation task, the test data generation rules, the test data distribution mode, and the information of the data storage middleware specified for distribution according to the test data configuration information input by the user. Then, according to the name or identification information of the required data structure, combined with the preset metadata warehouse, the logical structure and physical relationship of the data elements of the required data structure are further obtained, for example, through the data table name, combined with the preset metadata warehouse, the table structure, field type, primary key, foreign key and other information in the data table are further parsed.

[0067] Among them, the test data generation rules include but are not limited to test data generation logic (such as random generation, template-based generation, generation based on a specific distribution), test data generation range, test data constraints, associated user-defined extension scripts, etc.; the test data distribution mode is used to determine the specific distribution range of the test data, the timing of the distribution trigger, the time limit requirements of the distribution, etc., including but not limited to real-time distribution mode, batch timed distribution mode, event-triggered distribution mode, etc.

[0068] It should be noted that test data constraints refer to ensuring that the generated test data meets business rules or logical requirements, such as the uniqueness or format requirements for primary keys, field association requirements between multiple tables, field non-empty requirements, test data quantity requirements, etc. User-defined extension scripts refer to script files pre-written by users based on test data requirements to implement complex data generation logic. For example, user-defined extension scripts are scripts written by users to call external interfaces and process the fields of the results returned by the external interfaces.

[0069] It is understandable that in order to reduce the difficulty of writing custom scripts, the test data distribution device can also provide script templates and examples to testers, further reducing the difficulty of generating test data and improving the efficiency of database-related tests.

[0070] Step S130, constructing data conforming to the target data structure according to the data generation rule to obtain test data;

[0071] Step S140: Distribute the test data to the target storage middleware based on the target distribution mode.

[0072] In this embodiment, the test data distribution device generates test data that meets the test data generation range and test data constraints according to the target data structure information, the parsed test data generation logic or the associated user-defined extension script. Then, according to the data distribution mode configured by the user, that is, the target distribution mode, the test data generated above is automatically distributed to the data storage middleware configured by the user. It should be understood that in this application, the user can configure multiple target storage middleware at one time, so that the test data can be automatically distributed to multiple storage middleware after being generated once.

[0073] Furthermore, since there will always be basic data with a stable data structure or a data structure with a high usage frequency in the process of generating test data for the database, in order to further reduce the complexity of test data generation, before the above step S120, steps A01 to A02 may be included:

[0074] Step A01, identifying the target basic data source and extracting a number of data structure information;

[0075] Step A02: construct the metadata warehouse according to the plurality of data structure information.

[0076] It should be noted that the target basic data source is the basic data source of the required identification data structure set by the tester, usually including the basic data of the entire data storage system, and may also include the basic data of certain specified databases in the data storage system.

[0077] It is understandable that the target basic data source includes structured basic data sources, unstructured basic data sources, or basic data with only business significance. The test data distribution device needs to adopt different identification methods or technologies for the above-mentioned diverse basic data sources to extract multiple data structure information.

[0078] For example, for structured basic data sources, use automated tools to scan the target basic data sources, automatically identify and extract data structure information, including table structure, field type, primary key, foreign key, etc. For unstructured basic data sources, use data extraction technology to identify data patterns, that is, data structure information. For basic data that only has business meaning but no actual storage, field names, types, etc. can be manually constructed.

[0079] Then, the plurality of data structure information extracted above is stored in a certain designated data warehouse, thereby constructing a metadata warehouse.

[0080] The above implementation method identifies the target basic data source in advance and extracts multiple data structure information, which facilitates the reuse of the data structure for subsequent test data generation and reduces the time and cost of rebuilding the data structure.

[0081] The present embodiment provides a test data distribution method, which receives test data configuration information from a user; determines a target data structure, data generation rules, a target distribution mode, and a target storage middleware according to the test data configuration information and a preset metadata repository; constructs data that conforms to the target data structure according to the data generation rules to obtain test data; and distributes the test data to the target storage middleware based on the target distribution mode.

[0082] The present application receives the user's test data configuration information, and parses the test data configuration information and the preset metadata warehouse to obtain relevant information such as the target data structure, data generation rules, target distribution mode and target storage middleware; then, according to the data generation rules, constructs data that conforms to the target data structure to obtain test data, and distributes the test data to the target storage middleware based on the target distribution mode. This can achieve accurate generation of test data based on user test requirements, and automatically distribute test data to multiple storage middleware after it is generated once, effectively avoiding duplicate data generation in multi-source storage middleware, improving the efficiency of test data distribution between diverse data storage middleware, and further improving test efficiency.

[0083] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can refer to the above introduction, and will not be repeated later. Figure 2The step S130 further includes steps S210 to S240:

[0084] Step S210, using a preset script and according to the data generation rule, constructing data that conforms to the target data structure to obtain first constructed data;

[0085] Specifically, the test data distribution device runs a preset automation script file, parses the data generation rules, reads the test data generation logic, test data generation range, test data constraint related rules and parameter settings, and automatically constructs the test data according to the target data structure. Among them, the preset script is usually an automation script file written by relevant personnel in Python, Shell, SQL or other programming languages ​​based on the actual test data generation requirements, which is used to parse the data generation rules and construct data that meets the target data structure.

[0086] Furthermore, when the data generation rules include associated user-defined extension scripts, the user-defined extension scripts are run to parse the test data generation scope, test data constraints, and relevant rules and parameter settings of the target data structure, and automatically generate test data that meets the requirements.

[0087] Step S220, performing a data quality check on the first structured data based on a preset data quality check rule;

[0088] Step S230, when the data quality check fails, the user is fed back to reconfigure the test data;

[0089] Step S240: When the data quality check is qualified, the first constructed data is used as test data.

[0090] It can be understood that the preset data quality check rules refer to a series of preset standards and criteria for evaluating the quality of generated test data, which are pre-established by relevant personnel based on actual test data generation requirements and industry standards. The data quality check rules are used to define the quality requirements and characteristics that each data in the test data should meet. Data quality checks include but are not limited to data integrity checks, data validity checks, data consistency checks, business logic verification, etc.

[0091] Among them, data integrity check refers to checking whether the data is complete and there are no omitted or missing fields, records or information. The data integrity check rules usually require that the required fields of the data are not empty and that there are no outliers or missing values ​​in the data set.

[0092] Data validity check refers to checking whether the data is within the specified range and conforms to the predetermined format, such as whether the date is within a reasonable range, whether the numeric field is within the preset value range, etc.

[0093] Data consistency check refers to checking whether data is consistent across different storage systems, time periods, or data sources. Data consistency check rules usually include checking whether the naming, format, and unit of data fields are unified, and whether the logical relationships between data are consistent.

[0094] Business logic verification refers to verifying whether the test data complies with specific business rules or logic, such as the business amount cannot be a negative number, the business data generation time cannot be after the current day, etc.

[0095] Specifically, the test data distribution device performs a data check on the first construction data according to a preset data quality check rule to confirm whether the generated test data satisfies the data quality check rule.

[0096] Then, when the data quality check fails, the user needs to be notified that the test data generation failed, and the user needs to be informed to confirm whether the data quality check rules need to be adjusted, and then the test data configuration information is adjusted again, and the test data is generated and distributed again.

[0097] When the data quality check is qualified, it means that the first constructed data at this time can be directly used as test data for subsequent distribution in a variety of data storage middleware.

[0098] The present embodiment provides a test data distribution method, which receives test data configuration information from a user; determines a target data structure, data generation rules, a target distribution mode, and a target storage middleware according to the test data configuration information and a preset metadata repository; constructs data that conforms to the target data structure according to the data generation rules using a preset script to obtain first constructed data; performs a data quality check on the first constructed data based on preset data quality check rules; when the data quality check fails, provides feedback to the user to reconfigure the test data; when the data quality check passes, uses the first constructed data as test data; and distributes the test data to the target storage middleware based on the target distribution mode.

[0099] Through the above scheme, this embodiment uses a preset script to construct data that meets the target data structure according to the data generation rules obtained through analysis, thereby obtaining first constructed data; and then uses the preset data quality check rules to perform data quality check, thereby further ensuring the accuracy and consistency of the test data, and generating high-quality test data for subsequent distribution of the test data.

[0100] Based on the first embodiment and / or the second embodiment of the present application, in the third embodiment of the present application, the same or similar contents as those in the first embodiment and / or the second embodiment can be referred to the above description, and no further description will be given later. On this basis, the target storage middleware can be configured as several, please refer to Figure 3 The step S140 includes steps S310 to S330:

[0101] Step S310, confirming the data structure to be updated in the target storage middleware according to the target distribution mode;

[0102] Step S320, for each target storage middleware, performing corresponding adaptation conversion on the test data to obtain a plurality of adapted test data;

[0103] It should be noted that, in this embodiment, the target distribution mode is used to determine the data structure that the test data is used to update, including a full distribution mode and a partial distribution mode. Among them, the full distribution mode means that all data structures in the target storage middleware will be updated by the distributed test data; the partial distribution mode allows users to selectively update certain data structures in the target storage middleware.

[0104] Specifically, first, the test data distribution device determines the data structure to be updated in the target storage middleware according to the target distribution mode obtained by parsing the test data configuration information of the user.

[0105] Because different types of data storage middleware usually have inconsistent data formats, data type support, and performance optimization strategies, when the target storage middleware includes multiple, in order to ensure that the test data can be correctly stored and used in the target storage middleware, it is necessary to perform adaptability conversion of the test data for each target storage middleware, so as to obtain the corresponding several adapted test data. Among them, the adaptability conversion includes but is not limited to the adjustment of the data format, data type, character set, index, constraint, partition strategy and other aspects of the test data. For example, different data storage middleware supports different data formats. When the target storage middleware is HBASE, the test data needs to be converted into the format of {Table}{rowkey}{COL}{value}; when the target storage middleware is Redis, the test data needs to be processed into the format of {KEY}{value}. Or different data storage middleware supports different index creation rules and constraints. The test data needs to adjust the index and constraints according to the characteristics of the target storage middleware to optimize query performance and ensure data integrity.

[0106] Step S330: using a pre-created thread pool, synchronously and parallelly distribute the plurality of adapted test data to a plurality of target storage middlewares, and synchronously update the data structure to be updated.

[0107] Then, the test data distribution device uses the pre-created thread pool to synchronously distribute multiple adapted test data to the corresponding multiple target storage middlewares. Finally, the adapted test data is updated to the data structure to be updated, thereby completing the test data distribution to multiple data storage middlewares and ensuring the data consistency between multiple data storage middlewares.

[0108] Furthermore, in order to further improve the accuracy and consistency of test data generation, before the above step 140, steps B01 to B03 may be included:

[0109] Step B01, determining data validation rules and conflict resolution strategies according to the test data configuration information;

[0110] Step B02, based on the data verification rule, verify the test data to confirm whether there is a data conflict;

[0111] Step B03: when there is a conflict in the stored data, the test data is corrected based on a conflict resolution strategy.

[0112] Specifically, when inputting the test data configuration information, the user may also configure data validation rules and conflict resolution strategies for further validating the test data after the test data is generated, and checking and clearing invalid data in the test data.

[0113] First, the test data distribution device parses the test data configuration information to obtain data validation rules and conflict resolution strategies. Among them, the data validation rules are used to verify whether the execution results of the test data generation task meet the test requirements, including data uniqueness checks, data accuracy checks, data reliability checks, etc. of the generated test data; conflict resolution strategies refer to the rules and criteria for determining how to handle conflicts when data conflicts occur, such as when the primary key of the data is repeated, choose to merge the data, overwrite the data, or skip the data.

[0114] Among them, data uniqueness check refers to checking whether the records or field values ​​in the test data are unique to avoid duplicate or redundant data. For example, the data uniqueness check rule can be to ensure that the primary key field is unique in the data set. Data accuracy check refers to checking whether the number of data in the generated test data meets the requirements. Data reliability check refers to confirming the credibility of the test data by comparing it with known historical data, that is, judging whether the data is reliable and trustworthy.

[0115] Then, the test data distribution device applies data validation rules to the test data generated above to check whether the data in the test data meets the test requirements, whether there is duplicate or unreliable data, that is, invalid data, and deletes the invalid data using a conflict resolution strategy.

[0116] The present embodiment provides a test data distribution method, which receives test data configuration information from a user; determines a target data structure, data generation rules, a target distribution mode, and a target storage middleware according to the test data configuration information and a preset metadata repository; constructs data that conforms to the target data structure according to the data generation rules to obtain test data; confirms a data structure to be updated in the target storage middleware according to the target distribution mode; performs corresponding adaptation conversion on the test data for each target storage middleware to obtain a number of adapted test data; utilizes a pre-created thread pool to synchronously and parallelly distribute the several adapted test data to several target storage middleware, and synchronously updates the data structure to be updated.

[0117] Through the above scheme, this embodiment confirms the data structure to be updated in the target storage middleware according to the target distribution mode; then for each target storage middleware, the test data is adapted and converted accordingly, and the adapted test data is synchronously and parallelly distributed to the corresponding target storage middleware, which can improve the flexibility of test data distribution to meet the needs of different test scenarios, and realize the automatic distribution of test data to multiple storage middleware after one generation, improve the efficiency of test data distribution through synchronous and parallel distribution, and further improve the efficiency of testing.

[0118] Based on the third embodiment of the present application, in the fourth embodiment of the present application, the same or similar contents as those in the third embodiment can be referred to the above description, and will not be described in detail later. Figure 4 , after the above step S140, further comprising steps S410 to S420:

[0119] Step S410, performing a back check on the target storage middleware to confirm whether the test data is distributed successfully;

[0120] Step S420: When the test data distribution fails, the data is resent or reissued according to a preset compensation mechanism.

[0121] It can be understood that in order to ensure the accuracy and reliability of test data distribution, it is necessary to check the data in each target storage middleware after synchronous distribution to confirm the integrity and consistency of the test data in each target storage middleware. Specifically, by confirming whether the data in the target storage middleware is updated, the amount of updated data, the data content, etc., it is further confirmed whether the test data is distributed successfully.

[0122] When the result of the test data check is that the test data distribution fails, such as the test data transmission fails, the target middleware does not correctly receive the test data, etc., the test data distribution device needs to re-acquire the test data for resending or retransmitting according to the preset compensation mechanism. The preset compensation mechanism refers to the retry strategy for the test data distribution failure, including real-time, scheduled or batch resending to the target storage middleware that failed to distribute part of the test data, or when the test data distribution fails, directly distribute it to all target storage middleware in real time, scheduled or batch.

[0123] It should be understood that the test data distribution device can record logs generated during the test data distribution process. By analyzing the logs during the test data distribution process, potential problems can be quickly identified and solved, thereby improving the stability and reliability of the test data distribution device.

[0124] The present embodiment provides a test data distribution method, which receives test data configuration information from a user; determines a target data structure, data generation rules, a target distribution mode, and a target storage middleware according to the test data configuration information and a preset metadata repository; constructs data that conforms to the target data structure according to the data generation rules to obtain test data; distributes the test data to the target storage middleware based on the target distribution mode; performs a backcheck on the target storage middleware to confirm whether the test data has been successfully distributed; and when the test data distribution fails, resends or reissues the data according to a preset compensation mechanism.

[0125] Through the above scheme, this embodiment checks the target storage middleware to confirm whether the test data is distributed successfully, and then uses a preset compensation mechanism to handle the scenario of test data distribution failure, thereby further ensuring the reliability of test data distribution.

[0126] For example, in order to help understand the present application, the implementation process of the test data distribution method obtained after combining the above-mentioned embodiment 4 is as follows: Figure 5 , Figure 5 An example flow chart of a test data distribution method is provided, specifically:

[0127] exist Figure 5In the process, the data structure is first identified and established from the target basic data source in advance, and then the user configures the test data generation rules. The test data distribution device generates the basic data to be distributed (i.e., test data) according to the user's configuration information, and stores the basic data in a cache or database. Synchronously, the test data distribution device can check the basic data, and then provide feedback to the user to adjust the test data generation rules.

[0128] Then, according to the target middleware (ie, target storage middleware) to be distributed and the distribution strategy configured by the user, the test data distribution device synchronizes the basic data and sends it to the target middleware.

[0129] Finally, by back-checking the distribution results of the basic data, when the distribution result is a distribution failure, resending or retransmitting is performed.

[0130] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the test data distribution method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0131] This application also provides a test data distribution device, please refer to Figure 6 , the test data distribution device comprises:

[0132] Receiving module 10, used to receive test data configuration information of the user;

[0133] The parsing module 20 is used to determine the target data structure, data generation rules, target distribution mode and target storage middleware according to the test data configuration information and the preset metadata repository;

[0134] A data construction module 30, configured to construct data conforming to the target data structure according to the data generation rule to obtain test data;

[0135] The data distribution module 40 is used to distribute the test data to the target storage middleware based on the target distribution mode.

[0136] The test data distribution device provided by the present application adopts the test data distribution method in the above embodiment, which can solve the technical problem of how to efficiently and accurately distribute test data between diverse data storage middleware and reduce the complexity of test data generation. Compared with the prior art, the beneficial effects of the test data distribution device provided by the present application are the same as the beneficial effects of the test data distribution method provided by the above embodiment, and other technical features in the test data distribution device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0137] The present application provides a test data distribution device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the test data distribution method in the above-mentioned embodiment.

[0138] Reference below Figure 7 , which shows a schematic diagram of the structure of a test data distribution device suitable for implementing an embodiment of the present application. The test data distribution device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The test data distribution device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0139] like Figure 7As shown, the test data distribution device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the test data distribution device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the test data distribution device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a test data distribution device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided instead.

[0140] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0141] The test data distribution device provided by the present application adopts the test data distribution method in the above embodiment, which can solve the technical problem of how to efficiently and accurately distribute test data between diverse data storage middleware and reduce the complexity of test data generation. Compared with the prior art, the beneficial effects of the test data distribution device provided by the present application are the same as the beneficial effects of the test data distribution method provided by the above embodiment, and the other technical features in the test data distribution device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0142] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0143] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0144] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the test data distribution method in the above-mentioned embodiment.

[0145] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0146] The computer-readable storage medium may be included in the test data distribution device; or may exist independently without being assembled into the test data distribution device.

[0147] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the test data distribution device, the test data distribution device: receives the user's test data configuration information; determines the target data structure, data generation rules, target distribution mode and target storage middleware according to the test data configuration information and the preset metadata warehouse; constructs data that conforms to the target data structure according to the data generation rules to obtain test data; and distributes the test data to the target storage middleware based on the target distribution mode.

[0148] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0149] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0150] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0151] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned test data distribution method, and can solve the technical problem of how to efficiently and accurately distribute test data between diverse data storage middleware and reduce the complexity of test data generation. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the test data distribution method provided by the above-mentioned embodiment, and will not be repeated here.

[0152] An embodiment of the present application provides a computer program product, including a computer program, which implements the steps of the test data distribution method as described above when the computer program is executed by a processor.

[0153] The computer program product provided by the present application can solve the technical problem of how to efficiently and accurately distribute test data between diverse data storage middleware and reduce the complexity of test data generation. Compared with the prior art, the beneficial effects of the computer program product provided by the embodiment of the present application are the same as the beneficial effects of the test data distribution method provided by the above embodiment, which will not be repeated here.

[0154] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent processing scope of the present application.

Claims

1. A test data distribution method, characterized in that: The method comprises: Receive test data configuration information from the user; Determine the target data structure, data generation rules, target distribution mode and target storage middleware according to the test data configuration information and the preset metadata warehouse; According to the data generation rule, construct data that conforms to the target data structure to obtain test data; Based on the target distribution mode, the test data is distributed to the target storage middleware.

2. The method according to claim 1, characterized in that Before the step of determining the target data structure, data generation rules, target distribution mode and target storage middleware according to the test data configuration information and the preset metadata repository, the method includes: Identify the target basic data source and extract some data structure information; The metadata warehouse is constructed according to the plurality of data structure information.

3. The method according to claim 2, characterized in that The step of constructing data conforming to the target data structure according to the data generation rule to obtain test data comprises: Using a preset script and according to the data generation rule, construct data that conforms to the target data structure to obtain first constructed data; Performing a data quality check on the first structured data based on a preset data quality check rule; When the data quality check fails, the user is fed back to reconfigure the test data; When the data quality check is qualified, the first constructed data is used as test data.

4. The method according to claim 2, characterized in that The target storage middleware may be configured as a plurality of target storage middlewares. The step of distributing the test data to the target storage middlewares based on the target distribution mode includes: According to the target distribution mode, confirming the data structure to be updated in the target storage middleware; For each target storage middleware, the test data is adapted and converted accordingly to obtain a number of adapted test data; The plurality of adapted test data are synchronously and parallelly distributed to a plurality of target storage middlewares by using a pre-created thread pool, and the data structure to be updated is synchronously updated.

5. The method according to claim 2, characterized in that Before the step of distributing the test data to the target storage middleware based on the target distribution mode, the step further includes: Determine data validation rules and conflict resolution strategies based on the test data configuration information; Based on the data verification rules, verify the test data to confirm whether there is a data conflict; When the stored data conflicts, the test data is corrected based on the conflict resolution strategy.

6. The method according to any one of claims 2 to 5, characterized in that After the step of distributing the test data to the target storage middleware based on the target distribution mode, the method further includes: Performing a back check on the target storage middleware to confirm whether the test data is distributed successfully; When the test data distribution fails, the data is resent or reissued according to a preset compensation mechanism.

7. A test data distribution device, characterized in that: The test data distribution device comprises: A receiving module, used for receiving the test data configuration information of the user; A parsing module, used to determine the target data structure, data generation rules, target distribution mode and target storage middleware according to the test data configuration information and the preset metadata warehouse; A data construction module, used to construct data conforming to the target data structure according to the data generation rule to obtain test data; A data distribution module is used to distribute the test data to the target storage middleware based on the target distribution mode.

8. A test data distribution device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the test data distribution method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the test data distribution method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the test data distribution method according to any one of claims 1 to 6 are implemented.

Images