A method and device for constructing a database test environment

Abstract

The present invention discloses a method and device for constructing a database test environment, the construction method comprising: determining target database information that a user needs to create; obtaining a database image based on the target database information; determining target data that a user needs to store in a database; creating a data container of the target database based on the database image, and storing the target data that needs to be stored in a database based on the data container to complete the construction of the test environment. The embodiment of the present invention utilizes container creation technology to create a target database based on a database image selected by a user, and stores the target data that needs to be stored in a database based on the data container, thereby being able to build any sensitive data test environment, and realizing rapid construction of test environments for various databases, and is particularly suitable for the construction of sensitive data test environments.

Description

Technical Field

[0001] The present invention relates to the field of data security technology, and in particular to a method and device for constructing a database test environment. Background Art

[0002] In the prior art, various data security management platforms emerge in an endless stream. Compliant data security management platforms can classify and grade data, regardless of the medium in which the data is stored. Therefore, when testing the unified data management function and classification and grading function of the data security management platform, various database environments are required and the database also needs to contain various simulated sensitive test data.

[0003] There is no good method in the prior art to quickly build a test environment for various databases. Summary of the invention

[0004] The embodiment of the present invention provides a method and device for constructing a database test environment, which are used to quickly construct a test environment for various databases, and is particularly suitable for constructing a test environment for sensitive data.

[0005] The embodiment of the present invention provides a method for constructing a database test environment, comprising:

[0006] Determine the target database information that the user needs to create;

[0007] Obtain database image based on target database information;

[0008] Determine the target data that users need to store;

[0009] A data container of the target database is created based on the database image, and the target data to be stored is stored based on the data container to complete the construction of the test environment.

[0010] In some embodiments, the target database information includes the type and version of the target database;

[0011] Obtaining a database image based on the target database information includes:

[0012] Determine whether the local computer contains a database image of the corresponding type and version;

[0013] When the local database image does not contain a database image of the corresponding type and version, the database image is obtained from the public network.

[0014] In some embodiments, after determining the target data that the user needs to store, the construction method further includes:

[0015] Based on the target data that needs to be stored, determine whether relevant data needs to be generated in real time;

[0016] When it is determined that relevant data needs to be generated in real time, relevant data that conforms to the rules is generated by inversely deducing the regular expression.

[0017] In some embodiments, the target data that the user needs to store includes a target data set and the number of data items, and determining whether to generate relevant data in real time includes:

[0018] Determine whether the number of data entries in the current data set meets user requirements;

[0019] When the number of data items in the current data set does not meet user needs, it is determined that relevant data needs to be generated in real time.

[0020] In some embodiments, when it is determined that relevant data needs to be generated in real time, the construction method further includes:

[0021] Use extended regular expressions to generate related data of different data types.

[0022] In some embodiments, the target data is exported according to a preset format.

[0023] In some embodiments, the construction method further comprises:

[0024] Store multiple database images persistently locally;

[0025] Configure permissions for each user so that the user can manage the locally persistent database image based on the corresponding permissions.

[0026] The embodiment of the present invention further provides a system for constructing a database test environment, including a processor, which is configured as follows:

[0027] Determine the target database information that the user needs to create;

[0028] Obtain database image based on target database information;

[0029] Determine the target data that users need to store;

[0030] A data container of the target database is created based on the database image, and the target data to be stored is stored based on the data container to complete the construction of the test environment.

[0031] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for constructing a database test environment described in each embodiment of the present disclosure are implemented.

[0032] The embodiment of the present invention utilizes container creation technology to create a target database based on a database image selected by a user, and stores the target data to be stored in the database based on the data container, thereby being able to build any sensitive data test environment and quickly build a test environment for various databases, which is particularly suitable for building a sensitive data test environment.

[0033] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0035] Figure 1 A basic flow chart of a method for constructing a database test environment according to an embodiment of the present application;

[0036] Figure 2 This is a general flow chart of a method for constructing a database test environment according to an embodiment of the present application;

[0037] Figure 3 This is a schematic diagram of the basic framework of the system for building the database test environment for this application. DETAILED DESCRIPTION

[0038] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0039] The embodiment of the present invention provides a method for constructing a database test environment, such as Figure 1 As shown, the following steps are included:

[0040] In step S101, the target database information that the user needs to create is determined. Specifically, an input interface can be provided to determine the target database information that the user needs to create according to the type and version of the database that the user selects to create. The types of databases described in this example may include Mysql; Postgresql; Oracle-xe; Redis; MongoDB; MariaDB, etc., which are not specifically limited here.

[0041] In step S102, a database image is obtained based on the target database information. In some embodiments, the construction method further includes: locally persistently storing multiple database images, and configuring permissions for each user to enable the user to manage the locally persistent database image based on the corresponding permissions. Specifically, the image can also be tagged locally, and the image required by any user can be filtered through the tag. The image permissions can be managed by project, and the image permissions can also be managed by user. A certain image can be instantiated according to user needs, or a test container instantiated by the image can be repackaged into a new image.

[0042] In step S103, the target data that the user needs to store is determined. Specifically, a selection interface can be provided to the user, so as to determine the target data that the user needs to store according to the user's selection. The target data can be one or more sensitive test data sets, and each sensitive test data set includes test data of specified items.

[0043] In step S104, a data container of the target database is created based on the database image, and the target data to be stored is stored based on the data container to complete the construction of the test environment. In this example, the target database can be instantiated based on the acquired database image to create the target database, and the sensitive test data selected by the user is stored in the database, thereby completing the construction of the corresponding sensitive data.

[0044] The method of this example can generate various types of sensitive test data and test environments, such as ID number, mobile phone number, province, nationality, IP, MAC, taxpayer identification number, zip code, passport, education, license plate number, etc. The embodiment of the present invention uses container creation technology to create a target database based on a database image selected by a user, and stores the target data to be stored in the database based on the data container, so that any sensitive data test environment can be built, and the test environment of various databases can be quickly built, which is particularly suitable for the construction of sensitive data test environments.

[0045] In some embodiments, the target database information includes the type and version of the target database, and obtaining the database image based on the target database information includes: determining whether the local contains a database image of the corresponding type and version, for example, determining whether the local has a persistent database image based on the type and version of the database. Figure 2 As shown, when the local database image of the corresponding type and version is not included, the database image is obtained from the public network. In this specific example, adapters with various databases can be used to communicate with various databases. In this example, communicating with various databases means that various databases can be connected. For example, a connection tool such as Navicata can be used to store test data generated by the system. When the local database image of the corresponding type and version is not included, it is downloaded from the public network to meet the user's testing needs.

[0046] In some embodiments, after determining the target data that the user needs to store, the construction method further includes: judging whether it is necessary to generate relevant data in real time based on the target data that needs to be stored. In some embodiments, the target data that the user needs to store includes a target data set and the number of data entries, and judging whether it is necessary to generate relevant data in real time includes: judging whether the number of data entries in the current data set meets the user's needs; when the number of data entries in the current data set does not meet the user's needs, judging that it is necessary to generate relevant data in real time. That is, whether the number of data entries in the current data set meets the user's needs, in the case of judging that relevant data needs to be generated in real time, the relevant test data that conforms to the rules is generated by inverting the regular expression. When inverting the regular expression, the regular expression can also be tested to test whether the regular expression can normally invert the test data and generate a test case of the target number of entries to be echoed to the user, so that the user can judge whether the data type he wants is generated. If the data meets the user's requirements, the regular entry is verified, otherwise the verification fails. Additional data is generated by inverting the regular expression to complete, and the newly generated test data is added to the data set, thereby realizing the expansion of the data set. The method of the present application can quickly generate sensitive test data of any number of entries based on regular expressions, thereby reducing the time cost for R&D and testing personnel to find various types of test data.

[0047] In some embodiments, when it is determined that relevant data needs to be generated in real time, the construction method also includes: generating relevant test data of different data types by extending regular expressions. By extending regular expressions, more types of test data are generated, and a regular expression debugging page is provided to facilitate users to debug regular expressions to generate test data that meets expectations. The prior art cannot effectively expand the target data set of sensitive data. The method of the present application can expand the test data in the data set by using regular expressions and extended regular expressions, and the expanded data meets the expected requirements.

[0048] In some embodiments, the target data is exported according to a preset format, for example, the test data can be exported as CSV and JSON files to enrich the use scenarios of the test data.

[0049] The method for constructing the database test environment of the present application can be obtained from a data set, and test data that conforms to the regular expression can be generated by inverse regular expressions. When generating test data, when the total number of data entries exceeds the number of data entries built into the data set, additional data will be generated by inverse regular expressions to fill the gap, and the newly generated data will be added to the data set to expand the data set. By instantiating the database container through mirroring, then identifying the database type and version that needs to be adapted, and finally storing the data, it is possible to build a database environment containing sensitive test data. The method of the present application can use an interface to allow the user to input various parameters, and the system can automatically build a database test environment, reduce the user's learning cost, manage the database test environment throughout the system life cycle, and effectively utilize system hardware resources.

[0050] The embodiment of the present invention further provides a system for constructing a database test environment, including a processor, which is configured as follows:

[0051] Determine the target database information that the user needs to create;

[0052] Obtain database image based on target database information;

[0053] Determine the target data that users need to store;

[0054] A target database is created based on the database image, and target data to be stored is stored based on the data container, so as to complete the construction of the test environment.

[0055] like Figure 3 As shown, the construction system of the database test environment in this example

[0056] It contains three major management modules, namely system management module, data management module and image management module. The data are stored in three databases to facilitate the migration of various types of data. The specific functions of each module are described below:

[0057] The system management module includes:

[0058] User management submodule: This submodule is used to manage system users, such as adding, deleting, modifying and querying users.

[0059] Permission management submodule: This submodule is used to manage the permissions of each user and can limit the specific functional modules that each user can access.

[0060] Rule management submodule: This submodule is used to manage regular expressions, including adding, deleting, modifying, and checking the legitimacy of regular expressions.

[0061] Rule debugging submodule: This submodule is used to test whether the regular expression can normally infer the test data and generate a certain number of test cases to be displayed to the user, allowing the user to determine whether the desired data type is generated. If the data meets the user's requirements, the regular expression entry verification passes, otherwise the verification fails.

[0062] The data management module includes:

[0063] Data verification submodule: This submodule is used to verify whether the generated test data conforms to the regular

[0064] Data export submodule: This submodule is used to export test data to files, such as CSV files and JSON files; for example, the exported data format can be defined, such as exporting a CSV file containing N rows and N columns.

[0065] Data storage submodule: This submodule is used to store data and can dynamically adapt to various databases, including Mysql; Postgresql; Oracle-xe; Redis; MongoDB; MariaDB, etc.

[0066] Data generation submodule: This submodule is used to generate test data through regular expressions. Two generation methods are provided: one is to infer data that conforms to the regular expressions, and the other is to generate test data according to a preset set.

[0067] Image management module, including:

[0068] Image pulling submodule: This submodule supports searching images from the public network and pulling them locally.

[0069] Image instantiation submodule: This submodule is used to run the database container. Users can select the database type, database version, number of instances, and instance group to run.

[0070] Image export function submodule: This submodule is used to export the image as a tar file and provide it to users for download, allowing users to easily migrate each image.

[0071] Image import submodule: This submodule is used to import images from tar files into the system.

[0072] System database: stores system-related data and mainly interacts with system modules

[0073] Test data database: stores all kinds of generated test data and establishes relevant table structures according to user needs for easy reuse by users. The test database is completely separated from the system database, making it easy to migrate to other environments.

[0074] Mirror database: persistent storage of various database images. Images imported through the import module or downloaded through the image pull module will be stored in this database.

[0075] The construction system of this application can quickly build a system that meets the testing requirements of the data security management platform. The functions of the system include: being able to generate various simulated sensitive data through regular expressions, supporting the increase of generated data types by extending regular expressions, and supporting the export of data to files. Create virtual environments for various databases based on container technology and support expanding supported database types by uploading images. Able to automatically store simulated sensitive data. Support repackaging containers containing simulated sensitive data into images for easy migration.

[0076] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for constructing a database test environment described in each embodiment of the present disclosure are implemented.

[0077] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0078] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0079] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), including a number of instructions for a terminal (which can be a mobile phone, computer, server or network equipment, etc.) to execute the methods described in each embodiment of the present invention.

[0080] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation modes, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are within the protection of the present invention.

Claims

1. A method for constructing a database test environment, characterized in that: include: Determine the target database information that the user needs to create; Obtain database image based on target database information; Determine the target data that the user needs to store, where the target data is a sensitive test data set, and each sensitive data test set includes test data of specified items; Creating a data container of the target database based on the database image, and storing the target data to be stored in the database based on the data container to complete the construction of the test environment; After determining the target data that the user needs to store, the construction method further includes: Based on the target data that needs to be stored, determine whether relevant data needs to be generated in real time; When it is determined that relevant data needs to be generated in real time, relevant data that meets the rules is generated by inferring the regular expression; The target data that users need to store includes the target data set and the number of data entries. The determination of whether relevant data needs to be generated in real time includes: Determine whether the number of data entries in the current data set meets user requirements; When the number of data items in the current data set does not meet the user's needs, it is determined that relevant data needs to be generated in real time; Creating a data container of the target database based on the database mirror, and storing the target data to be stored in the database based on the data container includes: Instantiate the acquired database image to create a target database, and store the sensitive test data selected by the user into the created target database.

2. The method for constructing a database test environment according to claim 1, characterized in that: The target database information includes the type and version of the target database; Obtaining a database image based on the target database information includes: Determine whether the local computer contains a database image of the corresponding type and version; When the local database image does not contain a database image of the corresponding type and version, the database image is obtained from the public network.

3. The method for constructing a database test environment according to claim 1, wherein: In the case where it is determined that relevant data needs to be generated in real time, the construction method further includes: Use extended regular expressions to generate related data of different data types.

4. The method for constructing a database test environment according to claim 1, wherein: Export target data according to the preset format.

5. The method for constructing a database test environment according to claim 1, characterized in that: The construction method also includes: Store multiple database images persistently locally; Configure permissions for each user so that the user can manage the locally persistent database image based on the corresponding permissions.

6. A system for constructing a database test environment, characterized in that: The processor is configured as follows: Determine the target database information that the user needs to create; Obtain database image based on target database information; Determine the target data that the user needs to store, where the target data is a sensitive test data set, and each sensitive data test set includes test data of specified items; Creating a data container of the target database based on the database image, and storing the target data to be stored in the database based on the data container to complete the construction of the test environment; After determining the target data that the user needs to store, it also includes: Based on the target data that needs to be stored, determine whether relevant data needs to be generated in real time; When it is determined that relevant data needs to be generated in real time, relevant data that meets the rules is generated by inferring the regular expression; The target data that users need to store includes the target data set and the number of data entries. The determination of whether relevant data needs to be generated in real time includes: Determine whether the number of data entries in the current data set meets user requirements; When the number of data items in the current data set does not meet the user's needs, it is determined that relevant data needs to be generated in real time; Creating a data container of the target database based on the database mirror, and storing the target data to be stored in the database based on the data container includes: Instantiate the acquired database image to create a target database, and store the sensitive test data selected by the user into the created target database.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for constructing a database test environment according to any one of claims 1 to 5 are implemented.

Images