Data management method and device, equipment, medium and product

Abstract

The invention discloses a data management method and device, equipment, a medium and a product, and relates to the technical field of data test.The method comprises the steps that a data creation script is created based on preset service data; performing simulation data creation through the data creation script to obtain simulation data; and performing data management on the simulation data to obtain test data. Therefore, the simulation data is obtained by performing data generation on the business data obtained in advance, and then a series of data management is performed on the simulation data, so that the test data with high timeliness is obtained, the problem that the timeliness of the data used for testing is not high due to the fact that the simulation data cannot be managed is solved, and the accuracy of data management is improved.

Description

Technical Field

[0001] The present application relates to the field of data testing technology, and in particular to a data management method, device, equipment, medium and product. Background Art

[0002] Data testing is essential in software development and updating. The purpose of data testing is to ensure the quality and reliability of data, which is crucial for any data-driven decision and application.

[0003] However, at the current stage, test data is constructed by testers based on test requirements by writing scripts or preparing them manually, and is managed manually offline. The above test data management methods have the following disadvantages: (1) It is impossible to achieve centralized data management and sharing; (2) Multiple projects, multiple executors, automation and manual work are carried out simultaneously, and effective test data is often consumed and cannot be replenished in time; (3) The test data constructed by scripts or manually is not diverse enough and has certain differences from actual business data; (4) There are defects in the system under test, resulting in a large amount of dirty data in the test environment, which affects the test execution effect.

[0004] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0005] The main purpose of this application is to provide a data management method, device, equipment, medium and product, aiming to solve the technical problem that the simulation data cannot be managed, resulting in the low timeliness of the data used for testing.

[0006] To achieve the above purpose, the present application proposes a data management method, which is applied to a data generation end and includes:

[0007] Create a number-generating script based on preset business data;

[0008] Perform simulation number generation through the number generation script to obtain simulation data;

[0009] The simulation data is managed to obtain test data.

[0010] In one embodiment, the step of creating a number-generating script based on preset business data includes:

[0011] Performing feature recognition on the business data to obtain data features, where the data features include one or more of a data type and a data field;

[0012] According to the data features, a number-generating script is created by using a preset script creation tool.

[0013] In one embodiment, the data management includes one or more of data preservation, data quality detection, and data cleaning.

[0014] In one embodiment, before the step of performing data management on the simulation data, the method includes:

[0015] Analyze the business data to obtain data preservation requirements;

[0016] Analyze the data preservation requirements to obtain preservation trigger conditions, preservation environment, data filtering conditions, and preservation execution times;

[0017] The preservation configuration is obtained based on the preservation trigger condition, the preservation environment, the data filtering condition and the preservation execution times.

[0018] In one embodiment, the step of performing data management on the simulation data to obtain test data includes:

[0019] Performing data preservation on the simulation data based on the preservation configuration to obtain preserved data;

[0020] Performing data quality detection on the preservation data and the simulation data according to the data characteristics to obtain a quality detection result;

[0021] When the quality inspection result is passed, the fresh data is cleaned by using a preset dirty data tag to obtain test data.

[0022] In one embodiment, after the step of performing data management on the simulation data to obtain test data, the method includes:

[0023] Receive data sharing requests from the data testing platform;

[0024] Based on the data sharing request, setting a sharing tag and a sharing period corresponding to the test data;

[0025] According to the sharing tag and the sharing period, the test data is sent to the data testing platform, and the data is shared by the data testing platform.

[0026] In addition, to achieve the above purpose, the present application also proposes a data management device, the data management device comprising:

[0027] A creation module is used to create a number-generating script based on preset business data;

[0028] A number generation module, used for performing simulation number generation through the number generation script to obtain simulation data;

[0029] A management module is used to manage the simulation data to obtain test data.

[0030] In addition, to achieve the above objectives, the present application also proposes a data management device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the data management method described above.

[0031] In addition, to achieve the above objectives, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the data management method described above are implemented.

[0032] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, wherein the computer program product includes a computer program, and when the computer program is executed by a processor, the steps of the data management method described above are implemented.

[0033] One or more technical solutions proposed in this application have at least the following technical effects:

[0034] The data management method, device, equipment, storage medium and computer program product proposed in the embodiment of the present application are to obtain a number-generating script by creating a number-generating script based on preset business data; to obtain simulation data by simulating number-generating script; and to obtain test data by performing data management on the simulation data. Thus, simulation data is obtained by performing number-generating on the business data obtained in advance, and then a series of data management is performed on the simulation data to obtain test data with high timeliness, thereby solving the problem that the timeliness of the data used for testing is not high due to the inability to manage the simulation data, and improving the accuracy of data management. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0036] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0037] Figure 1 A flowchart of the first embodiment of the data management method of this application is provided;

[0038] Figure 2This is a schematic diagram of the field association relationship involved in the data management method of this application;

[0039] Figure 3 A schematic diagram of data characteristics of simulation data involved in the data management method of this application;

[0040] Figure 4 This is a schematic diagram of the data management method for this application involving feature extraction of enumeration fields;

[0041] Figure 5 This is a schematic diagram of feature extraction of amount value fields involved in this application data management method;

[0042] Figure 6 A schematic diagram of the data features recommended for the data management method of this application;

[0043] Figure 7 The data management method of this application involves a schematic diagram of a data canvas obtained after the number-generating script generates numbers;

[0044] Figure 8 A flowchart of the second embodiment of the data management method of this application is provided;

[0045] Fig. 9 This is a schematic diagram of the data preservation configuration involved in the data management method of this application;

[0046] Fig.10 This is a schematic diagram of the data quality review involved in the data management method for this application;

[0047] Fig.11 This is a schematic diagram of the data management method for this application involving data sharing and locking;

[0048] Fig.12 This is a schematic diagram of the module structure of the data management device according to an embodiment of the present application;

[0049] Fig.13 Schematic diagram of the device structure of the hardware operating environment involved in the data management method in the embodiment of the present application.

[0050] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0051] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0052] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0053] The main solution of the embodiment of the present application is: perform feature recognition on the business data to obtain data features, and the data features include one or more of data types and data fields; according to the data features, create a number-making script through a preset script creation tool. The data management includes one or more of data preservation, data quality detection and data cleaning. Parse the business data to obtain data preservation requirements; analyze the data preservation requirements to obtain preservation trigger conditions, preservation environment, data filtering conditions and preservation execution times; configure based on the preservation trigger conditions, preservation environment, data filtering conditions and preservation execution times to obtain preservation configuration. Based on the preservation configuration, perform data preservation on the simulation data to obtain preservation data; perform data quality detection on the preservation data and simulation data according to the data features to obtain quality detection results; when the quality detection result is passed, perform data cleaning on the preservation data through a preset dirty data tag to obtain test data. Receive a data sharing request from a data testing platform; based on the data sharing request, set a sharing tag and a sharing period corresponding to the test data; according to the sharing tag and the sharing period, send the test data to the data testing platform, and the data testing platform will share the data. This solves the problem of low timeliness of data used for testing due to the inability to manage simulation data, realizes data management, and improves the timeliness of test data. Based on the scheme of the present invention, from the reality that the existing test data in the current stage is constructed by testers based on test requirements by writing scripts or manually preparing, and offline manual management is adopted, the existing test data management method has the following disadvantages: offline management of test data cannot realize centralized management and sharing of data, multiple projects, multiple executors, automation and manual work are carried out simultaneously, and effective test data is often consumed and cannot be replenished in time. In addition, the test data constructed by scripts or manual work is not diverse enough and there are certain differences from the actual business data. Furthermore, there are defects in the system under test, which leads to a large amount of dirty data in the test environment, affecting the test execution effect, thereby the timeliness of the test data is low. Starting from the problem, a data management method is designed, and the effectiveness of the data management method of the present invention is verified when identifying user intentions. Finally, the timeliness of the test data obtained by data management using the method of the present invention is significantly improved.

[0054] In this embodiment, for the convenience of description, the following description is made with the data management device as the execution subject.

[0055] Since data testing is indispensable in software development and updating, the purpose of data testing is to ensure the quality and reliability of data, which is crucial for any data-driven decision and application. However, at the current stage, test data is constructed by testers based on test requirements through script writing or manual preparation, and is managed manually offline. The above test data management method has the following disadvantages: (1) It is impossible to achieve centralized management and sharing of data. (2) Multiple projects, multiple executors, automation and manual work are carried out simultaneously, and effective test data is often consumed and cannot be replenished in time. (3) The test data constructed by scripts or manual work is not diverse enough and there are certain differences between it and the actual business data. (4) There are defects in the system under test, resulting in a large amount of dirty data in the test environment, which affects the timeliness of the test data and reduces the test execution effect.

[0056] The present application provides a solution, in which simulation data is generated by using pre-acquired business data in the number-generating end, and then a series of data management is performed on the simulation data, so as to obtain test data with high timeliness and provide users with better services.

[0057] From the above embodiments, it can be seen that the present application obtains a number-generating script by creating a number-generating script based on preset business data; simulates the number-generating script to obtain simulation data; and performs data management on the simulation data to obtain test data. Thus, simulation data is obtained by performing number generation on the business data obtained in advance, and then a series of data management is performed on the simulation data to obtain test data with high timeliness, which solves the problem that the timeliness of the data used for testing is not high due to the inability to manage the simulation data, and improves the accuracy of data management.

[0058] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device, an intention device device, etc. that can realize the above functions. The following takes the data management device as an example to illustrate this embodiment and the following embodiments.

[0059] Based on this, the present application embodiment provides a data management method, referring to Figure 1 , Figure 1 This is a flowchart of the first embodiment of the data management method of this application.

[0060] In this embodiment, the data management method is applied to the data generation end, and the method includes steps S01 to S06:

[0061] Step S01, creating a number-generating script based on preset business data;

[0062] It should be clear that the solution of this embodiment mainly addresses the problem that data testing is indispensable in software development and updating. The purpose of data testing is to ensure the quality and reliability of data, which is crucial for any data-driven decision and application. However, at the current stage, test data is constructed by testers based on test requirements by writing scripts or preparing them manually, and is managed manually offline. The above test data management method has the following disadvantages: (1) It is impossible to achieve centralized management and sharing of data; (2) Multiple projects, multiple executors, automation and manual work are carried out simultaneously, and effective test data is often consumed and cannot be replenished in time; (3) The test data constructed by scripts or manually is not diverse enough and there are certain differences between it and the actual business data; (4) There are defects in the system under test, resulting in a large amount of dirty data in the test environment, which affects the test execution effect.

[0063] Therefore, the main purpose of this embodiment is to create numbers based on business data, and the business data is obtained from the business system in advance. It can be obtained in real time when the business system is processing business, or it can be obtained by intercepting data after the business system completes the processing. The specific business system can be loan business, credit card business, savings card business, etc., and the business data can directly reflect the trend of the current business, so the necessity of conducting business testing on it is becoming more and more important. In this embodiment, a script is created based on the pre-obtained business data, and a number-creating script for creating business data is obtained.

[0064] Step S02, performing simulation number generation through the number generation script to obtain simulation data;

[0065] After obtaining the number-generating script, simulate the number-generating script to make the data in the test environment close to production, improve the test coverage and test efficiency. When generating numbers, you can configure the number-generating function for each field, use the data canvas to call the general function, SQL, API or Python encapsulated function, and reference the fields of this table or other tables when generating numbers, such as Figure 2 As shown, the association between the fields is guaranteed, and the page of simulation data is obtained as follows Figure 3 As shown, it includes various features of the simulation data, such as the main words of the rental phase information in the simulation data of the lending business, the rental UID, the rental version number, the phase serial number, the original currency principal, the phase repayment principal, the phase financing amount, the phase rent calculation method, etc.

[0066] Step S06: performing data management on the simulation data to obtain test data.

[0067] After obtaining the simulation data, it is necessary to manage it accordingly. It should be clear that in order to ensure the timeliness of the test data, the data management in this embodiment includes but is not limited to data preservation, data quality detection and data cleaning.

[0068] Among them, data preservation refers to maintaining the freshness and validity of data, which includes regular updates and verifications to ensure that the data always reflects the latest reality. Data quality management strategies can ensure that the data is accurate and reliable when used, and common data cleaning methods include cleaning up outdated or duplicate data.

[0069] The advantages of data quality testing are (1) accuracy: verifying whether the data is correct, (2) completeness: ensuring that there are no missing data, (3) consistency: checking the consistency of data in different systems, (4) validity: confirming that the data conforms to predefined rules and formats, (5) timeliness: whether the data is updated in a timely manner to reflect the latest information;

[0070] Data cleaning includes the following methods: (1) Deduplication: Identify and remove duplicate records in the data set to avoid redundancy and confusion, (2) Fill in missing values: Process the missing parts of the data, which may be completed through interpolation, mean value filling, data inference, etc., (3) Correct data errors: Correct errors in the data, such as spelling errors, inconsistent formats, invalid values, etc., (4) Standardize data formats: Unify the data into a standard format to ensure consistency, for example, unify the date format to "YYYY-MM-DD", (5) Remove invalid data: Remove data records that are no longer useful or outdated to maintain the relevance and validity of the data set, (5) Process Outliers: Identify and handle outliers in the data to ensure that they do not affect the analysis results. Outliers may require further investigation to determine whether they need to be adjusted or deleted. (6) Verify data integrity: Ensure that the various parts of the data are related to each other and conform to business rules and logical relationships. (7) Convert data types: Convert data to appropriate data types for easy analysis and processing, such as converting strings to date types. (8) Normalize data: Normalize data when necessary to facilitate comparison and analysis, such as converting values ​​to the same range. (9) Data integration: Merge data from different sources to ensure data consistency and integrity.

[0071] Specifically, the above step S01, the step of creating a number-generating script based on preset business data, includes:

[0072] Step S011, performing feature recognition on the business data to obtain data features, where the data features include one or more of a data type and a data field;

[0073] Step S012: creating a number-generating script using a preset script creation tool according to the data features.

[0074] Feature recognition is performed on business data to obtain the data features corresponding to the business data, and the corresponding data types and data fields are obtained. Business fields are usually divided into amount type, value type, date type, text type, and enumeration type. For each type of business field, the features to be extracted have slightly different focuses. For example: (1) For enumeration type fields, the focus is on the data distribution of the enumeration interval and enumeration value. For traditional financial business, the value of the enumeration field almost represents a business scenario. If a value outside the enumeration range appears during data feature analysis, a new business scenario is understood. In addition, by analyzing the distribution of enumeration values, high-frequency business scenarios, low-frequency business scenarios, normal business scenarios, and abnormal business scenarios can be identified, such as Figure 4 As shown in the figure, the currency type is an enumeration field that the tester needs to confirm whether it appears. By checking the distribution of the currency enumeration value, it can be analyzed that a certain credit business is mainly RMB business, with less USD business, and almost no Euro, Yen and other businesses. (2) Amount and value fields such as Figure 5 As shown, focus on the maximum value, minimum value, precision, whether it is empty and other features. Amount fields usually represent the size of the business. It is necessary to test the system boundary value based on the amount range and precision. (3) Date fields focus on the offset value of the date. The date field can be used to analyze the time when the business and transaction occurred and determine the high-frequency transaction time. (4) Text fields focus on the text length, whether there is an empty value and other features to test the system boundary value.

[0075] Got Figure 6 After the data features shown are obtained, they can be constructed using a variety of tools and programming languages. Common ones include: (1) Python: Generate random data using the Faker library or using custom code; (2) JavaScript: Create simulated data in the front-end or Node.js environment; (3) R: Generate data using packages such as dplyr and tidyverse; (4) SQL: Generate test data using functions such as GENERATE_SERIES in the database; (5) Excel / VBA: Generate simulated data using Excel macros. Specific number-generating scripts are scripts used to generate simulated data or test data. Their uses include: (1) Testing and development: Create a large amount of test data to verify system functions and performance; (2) Data analysis: Generate sample data for analyzing the effectiveness of the algorithm; (3) Demonstration and training: Provide data for demonstration and training purposes to obtain the corresponding number-generating scripts;

[0076] Then you can simulate the number generation based on the number generation script, through Figure 7The data canvas shown obtains various information of the data.

[0077] This embodiment uses the above scheme to create a number-generating script based on preset business data; simulate numbers through the number-generating script to obtain simulation data; and manage the simulation data to obtain test data. Thus, simulation data is generated through the business data obtained in advance, and then a series of data management is performed on the simulation data to obtain test data with high timeliness, which solves the problem that the timeliness of the data used for testing is not high due to the inability to manage the simulation data, and improves the accuracy of data management.

[0078] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 8 Before step S06, before the step of performing data management on the simulation data, the method further includes steps S03 to S05:

[0079] Step S03, parsing the business data to obtain data preservation requirements;

[0080] Step S04, analyzing the data preservation requirement to obtain a preservation trigger condition, a preservation environment, a data filtering condition, and a preservation execution frequency;

[0081] Step S05, configuring to obtain a preservation configuration based on the preservation trigger condition, the preservation environment, the data filtering condition and the preservation execution times.

[0082] In this embodiment, before preserving the data, the business data needs to be parsed to obtain the data preservation requirements corresponding to the business data, where the data preservation requirements include but are not limited to preservation trigger conditions, preservation environment, data filtering conditions, and preservation execution times;

[0083] Then, the preservation configuration is obtained based on the preservation trigger condition, preservation environment, data filtering condition, and preservation execution times, such as Fig. 9 As shown in the figure, data preservation requires the following configuration elements:

[0084] (1) Trigger condition configuration, including timed trigger and conditional trigger: Timed trigger requires setting Cron expression, which can be generated with AI assistance. Conditional trigger requires configuring preservation conditions and thresholds. Currently, three conditions are supported: SQL encapsulation query result is less than the threshold, the number of table data entries is less than the threshold, and the number of SQL query result entries is less than the threshold.

[0085] (2) Fresh environment: Supports simultaneous freshness preservation of multiple environments such as ST, UAT, and DEV;

[0086] (3) Conditions for participating in preservation: You can enter the where filter conditions for the data participating in preservation, such as id>1; if no conditions are set, all data will be preserved by default;

[0087] (4) Data cleaning conditions: Supports four data cleaning strategies, including no cleaning, cleaning before preservation, cleaning after preservation, and cleaning all preservation data;

[0088] (5) Failure retries: The number of retries after a number generation task fails to execute a number generation expression in each row;

[0089] (6) Failure strategy: When the number generation fails, the system supports ignoring the row of data and continuing the number generation, or directly exiting the number generation task;

[0090] (7) Number-generating task: Select the simulation number-generating task configured in step 2;

[0091] (8) Select the table name in the number configuration: the database table name in the number configuration;

[0092] (9) Execution times: set the number of times data is kept fresh;

[0093] By configuring the above elements, data can be preserved so that the business data distribution meets the needs of manual and automated testing.

[0094] After knowing the above preservation configuration, the data can be managed accordingly. Specifically, the above step S06, the step of managing the simulation data to obtain test data includes:

[0095] Step S061, preserving the simulation data based on the preservation configuration to obtain preserved data;

[0096] Step S062, performing data quality detection on the preservation data and the simulation data according to the data characteristics to obtain a quality detection result;

[0097] Step S063, when the quality inspection result is passed, clean the fresh data using a preset dirty data tag to obtain test data.

[0098] Firstly, the simulation data is preserved based on the previously obtained preservation configuration to obtain the corresponding preservation data;

[0099] Then, if Fig.10 As shown, by comparing the feature analysis results, the data features and the latest differences after data creation and data preservation are checked, and the quality of the test data is checked back to obtain the quality test results;

[0100] Finally, dirty data is labeled through data feature analysis and manual demarcation, and scheduled tasks for data cleaning are configured to clean dirty data regularly to improve the success rate of automated execution.

[0101] After obtaining the test data, it is also necessary to share the test data so that the test data can be used for the final data test. Therefore, after the above step S06, in which the simulation data is managed to obtain the test data, the method includes:

[0102] Step S07, receiving a data sharing request from a data testing platform;

[0103] Step S08, based on the data sharing request, setting a sharing tag and a sharing period corresponding to the test data;

[0104] Step S09: sending the test data to the data test platform according to the sharing tag and the sharing period, and the data test platform performs data sharing.

[0105] After obtaining the test data, it is also necessary to send the test data to the corresponding data test terminal.

[0106] like Fig.11 As shown, this embodiment provides a data sharing and data locking mechanism to solve the problem of repeated data preparation between testers and teams, and solves the problem of multiple people and multiple scripts using shared data through the data locking mechanism;

[0107] Shared data refers to the data shared by users from the data pool of this product. Shared data can be set up at multiple sharing levels, such as center level and team level. Center-level shared data can be used by all products in the center, while team-level shared data can only be used within the team. Shared data can also support labeling, which helps users quickly find the data they need by labeling shared data. Shared data supports setting a sharing period, and shared data will be automatically released after the expiration date. After release, it cannot be used by other users or scripts.

[0108] In addition, shared data supports a locking function. The purpose of locking is to prevent other users or scripts from modifying, deleting or otherwise modifying it. For example, a batch of automated test data is prepared by making numbers. In order to ensure the success rate and accuracy of automated execution, this batch of data needs to be locked in advance to prevent the data from being tampered with by others. Furthermore, in this embodiment, the data can also be locked with a time limit. The data locking and unlocking operations can be achieved by setting the locking time.

[0109] This embodiment uses the above scheme to obtain data preservation requirements by parsing the business data; analyzing the data preservation requirements to obtain preservation trigger conditions, preservation environment, data filtering conditions, and preservation execution times; and configuring based on the preservation trigger conditions, preservation environment, data filtering conditions, and preservation execution times to obtain preservation configuration. Thus, simulation data is generated by generating numbers using the business data obtained in advance, and then a series of data management is performed on the simulation data to obtain test data with high timeliness, which solves the problem of low timeliness of data used for testing due to the inability to manage simulation data and improves the accuracy of data management.

[0110] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the data management method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0111] This application also provides a data management device, please refer to Fig.12 , the data management is applied to the number-generating end, and the device includes:

[0112] A creation module 10 is used to create a number-generating script based on preset business data;

[0113] A number generation module 20, used for performing simulation number generation through the number generation script to obtain simulation data;

[0114] The management module 30 is used to manage the simulation data to obtain test data.

[0115] The data management device provided by the present application adopts the data management method in the above embodiment, which can solve the technical problem that the simulation data cannot be managed, resulting in the low timeliness of the data used for testing. Compared with the prior art, the beneficial effects of the data management device provided by the present application are the same as the beneficial effects of the data management method provided by the above embodiment, and the other technical features in the data management device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0116] The present application provides a data management device, which includes: at least one processor; and a memory that is communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the data management method in the above-mentioned embodiment one.

[0117] Reference below Fig.13, which shows a schematic diagram of the structure of a data management device suitable for implementing the embodiment of the present application. The data management device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig.13 The data management device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0118] like Fig.13 As shown, the data management device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the data management device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the data management device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a data management device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0119] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0120] The data management device provided by the present application adopts the data management method in the above embodiment, which can solve the technical problem that the simulation data cannot be managed, resulting in the low timeliness of the data used for testing. Compared with the prior art, the beneficial effects of the data management device provided by the present application are the same as the beneficial effects of the data management method provided by the above embodiment, and the other technical features in the data management device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0121] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0122] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0123] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the data management method in the above-mentioned embodiment.

[0124] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0125] The computer-readable storage medium may be included in the data management device; or may exist independently without being installed in the data management device.

[0126] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the data management device, the data management device: creates a number-generating script based on preset business data; performs simulation number generation through the number-generating script to obtain simulation data; and performs data management on the simulation data to obtain test data.

[0127] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0128] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0129] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0130] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned data management method, and can solve the technical problem that the simulation data cannot be managed, resulting in low timeliness of the data used for testing. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the data management method provided by the above-mentioned embodiment, and will not be elaborated here.

[0131] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned data management method when executed by a processor.

[0132] The computer program product provided by the present application can solve the technical problem that the simulation data cannot be managed, resulting in low timeliness of the data used for testing. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the data management method provided by the above embodiment, which will not be described in detail here.

[0133] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A data management method, It is characterized in that The method is applied to a number generating end, and the method comprises: Create a number-generating script based on preset business data; Perform simulation number generation through the number generation script to obtain simulation data; The simulation data is managed to obtain test data.

2. The method according to claim 1, It is characterized in that The step of creating a number-generating script based on preset business data includes: Performing feature recognition on the business data to obtain data features, where the data features include one or more of a data type and a data field; According to the data features, a number-generating script is created by using a preset script creation tool.

3. The method according to claim 1, It is characterized in that The data management includes one or more of data preservation, data quality detection and data cleaning.

4. The method according to claim 3, It is characterized in that Before the step of performing data management on the simulation data, the method includes: Analyze the business data to obtain data preservation requirements; Analyze the data preservation requirements to obtain preservation triggering conditions, preservation environment, data filtering conditions, and preservation execution times; The preservation configuration is obtained based on the preservation trigger condition, the preservation environment, the data filtering condition and the preservation execution times.

5. The method according to claim 4, It is characterized in that The step of performing data management on the simulation data to obtain test data comprises: Performing data preservation on the simulation data based on the preservation configuration to obtain preserved data; Performing data quality detection on the preservation data and the simulation data according to the data characteristics to obtain a quality detection result; When the quality inspection result is passed, the fresh data is cleaned by using a preset dirty data tag to obtain test data.

6. The method according to claim 1, It is characterized in that After the step of performing data management on the simulation data to obtain test data, the method includes: Receive data sharing requests from the data testing platform; Based on the data sharing request, setting a sharing tag and a sharing period corresponding to the test data; According to the sharing tag and the sharing period, the test data is sent to the data testing platform, and the data is shared by the data testing platform.

7. A data management device, It is characterized in that The device is applied to a number generating end, and comprises: A creation module is used to create a number-generating script based on preset business data; A number generation module, used for performing simulation number generation through the number generation script to obtain simulation data; A management module is used to manage the simulation data to obtain test data.

8. A data management device, It is characterized in that The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the data management method according to any one of claims 1 to 6.

9. A storage medium, It is characterized in that The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the data management method according to any one of claims 1 to 6 are implemented.

10. A computer program product, It is characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the data management method according to any one of claims 1 to 6 are implemented.

Images