Test data verification method, verification device and computer readable storage medium
By creating and automatically mounting logical volumes in a distributed storage system, generating data verification tools, and verifying the reliability of the application business model, the problem of low verification efficiency of test data consistency in the existing technology is solved, and high efficiency is achieved Logical volume verification and real business scenario simulation.
Patent Information
- Application Number
- CN202010797862.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-08-10
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2040-11-12
AI Technical Summary
The existing test data consistency verification methods are inefficient, and it is impossible to directly verify logical bare volumes. It is time-consuming and resource-consuming to verify formatted files of larger-capacity logical volumes, so it is impossible to simulate real business scenarios.
Create a logical volume in a distributed storage system and automatically mount it to the application terminal, generate a data verification tool, determine the reliability of the application business model by verifying the consistency of writing and reading data, and simulate hardware failures to verify data consistency sex.
The logical volume verification process is simplified, the verification efficiency is improved, and the simulation of a variety of real business and load conditions is realized, verification time is saved, and the logical naked volume can be directly detected, and consistency problem logs are generated to locate software defects.
Smart Images

Figure CN111949436B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing, and in particular to a test data verification method, a verification device, a computer-readable storage medium, and a processor. Background Art
[0002] Software-defined distributed storage is a trend in the industry. Due to the advantages of distributed storage architecture, hardware can be infinitely expanded to provide greater capacity and better performance, so it can carry and run a variety of different applications. Most of the existing distributed storage uses multiple copies and erasure coding technology to ensure high reliability and high availability of the system. However, the distributed storage system is composed of a large number of servers, hard disks, and networks, so hardware damage will affect the reliability and availability of the distributed storage system. Therefore, it is particularly important to quickly verify the consistency of test data for the distributed storage system.
[0003] The existing verification method for testing data consistency generally adopts MD5 value verification. For the logical volume in the storage system, the file system needs to be formatted for verification. The logical bare volume cannot be directly verified, and the verification process is complicated. Moreover, for the large-capacity logical volume formatting file verification is time-consuming and consumes system resources, and it cannot form a coherent and continuous comparison verification, and cannot simulate real business scenarios.
[0004] The above information disclosed in the background technology section is only used to enhance the understanding of the background technology of the technology described in this article. Therefore, the background technology may contain certain information that does not form the prior art known in this country for those skilled in the art. Summary of the invention
[0005] The main purpose of the present application is to provide a test data verification method, a verification device, a computer-readable storage medium and a processor to solve the problem of low verification efficiency of the test data consistency verification method in the prior art.
[0006] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a method for verifying test data is provided, including: configuring an application business model, wherein the application business model is used to simulate real business and the load conditions of the real business; writing the application business model into a distributed storage system; creating multiple logical volumes for the application business model in the distributed storage system; automatically mounting the multiple logical volumes to an application terminal; generating a data verification tool in the application terminal; and using the data verification tool to verify the reliability of the application business model when running in the distributed storage system.
[0007] Optionally, applying the data verification tool to verify the reliability of the application business model when running in the distributed storage system includes: applying the data verification tool to verify the consistency of write data and read data, the write data being data in the application business model written into the distributed storage system, and the read data being data written to the application terminal after the write data is read from the distributed storage system; when the write data and the read data are consistent, determining that the application business model is reliable when running in the distributed storage system; when the write data and the read data are inconsistent, determining that the application business model is unreliable when running in the distributed storage system.
[0008] Optionally, the data verification tool is used to verify the consistency of written data and read data, including: applying the data verification tool to process the written data in blocks to obtain multiple first verification data structures; simulating a hardware failure; in the process of simulating the hardware failure, obtaining the read data; applying the data verification tool to process the read data in blocks to obtain multiple second verification data structures; and verifying the consistency of the first verification data structure with the corresponding second verification data structure.
[0009] Optionally, verifying the consistency of the first verification data structure with the corresponding second verification data structure includes: obtaining the first verification part of each of the first verification data structures; obtaining the second verification part of each of the second verification data structures; and verifying the consistency of the first verification part with the second verification part.
[0010] Optionally, when the written data and the read data are consistent, determining that the application business model is reliable when running in the distributed storage system includes: when the first verification unit is consistent with the second verification unit, determining that the application business model is reliable when running in the distributed storage system.
[0011] Optionally, when the written data and the read data are inconsistent, after determining that the application business model is unreliable when running in the distributed storage system, the verification method further includes: the data verification tool exits and generates a consistency problem log; determining a problematic logical address in the logical volume based on the consistency problem log; and determining a software defect in the distributed storage system based on the logical address.
[0012] Optionally, configuring the application business model includes: applying a python script to automatically configure the application business model.
[0013] Optionally, automatically mounting the plurality of logical volumes to an application terminal includes: applying a shell or python script to automatically mount the plurality of logical volumes to the application terminal.
[0014] According to another aspect of the present application, a test data verification device is provided, including: a configuration unit, used to configure an application business model, wherein the application business model is used to simulate real business and the load conditions of the real business; a writing unit, used to write the application business model into a distributed storage system; a creation unit, used to create multiple logical volumes in the distributed storage system for the application business model; a mounting unit, used to automatically mount the multiple logical volumes to an application terminal; a first generation unit, used to generate a data verification tool in the application terminal; and a verification unit, used to apply the data verification tool to verify the reliability of the application business model when running in the distributed storage system.
[0015] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute any one of the test data verification methods.
[0016] According to another aspect of the present application, a processor is provided, wherein the processor is used to run a program, wherein the program executes any one of the test data verification methods when running.
[0017] By applying the technical solution of the present application, a logical volume is created for the corresponding application business model in a distributed storage system, and then the created logical volume is automatically mounted to the application terminal, a data verification tool is generated in the application terminal, and the data verification tool is used to verify the reliability of the application business model when it is running in the distributed storage system. By automatically mounting the logical volume to the application terminal, the verification of the logical bare volume is realized, that is, there is no need to format the logical bare volume before verification, which simplifies the verification process of the logical volume, improves the verification efficiency, realizes the simulation of various real businesses and real business load conditions, and saves data verification time through automatic configuration of the application business model and automatic mounting of the logical volume. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] The drawings constituting part of the present application are used to provide a further understanding of the present application. The exemplary embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0019] Figure 1 A flow chart showing a method for verifying test data according to an embodiment of the present application; and
[0020] Figure 2 A schematic diagram of a device for verifying test data according to an embodiment of the present application is shown. DETAILED DESCRIPTION
[0021] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0022] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.
[0023] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0024] It should be understood that when an element (such as a layer, film, region, or substrate) is described as being "on" another element, the element may be directly on the other element, or there may be intermediate elements. Moreover, in the specification and claims, when an element is described as being "connected" to another element, the element may be "directly connected" to the other element, or "connected" to the other element through a third element.
[0025] As introduced in the background technology, the verification efficiency of the test data consistency verification method in the prior art is low. In order to solve the above technical problems, the embodiments of the present application provide a test data verification method, a verification device, a computer-readable storage medium and a processor.
[0026] According to an embodiment of the present application, a method for verifying test data is provided.
[0027] Figure 1 1 is a flow chart of a method for verifying test data according to an embodiment of the present application. Figure 1 As shown, the method comprises the following steps:
[0028] Step S101, configuring an application business model, where the application business model is used to simulate real business and the load of the real business;
[0029] Step S102, writing the above application business model into the distributed storage system;
[0030] Step S103, creating multiple logical volumes in the distributed storage system for the application business model;
[0031] Step S104, automatically mounting the plurality of the above logical volumes to the application terminal;
[0032] Step S105, generating a data verification tool in the above application terminal;
[0033] Step S106: Use the data verification tool to verify the reliability of the application business model when it runs in the distributed storage system.
[0034] In the above scheme, a logical volume is created for the corresponding application business model in the distributed storage system, and then the created logical volume is automatically mounted to the application terminal, a data verification tool is generated in the application terminal, and the data verification tool is used to verify the reliability of the application business model when it is running in the distributed storage system. By automatically mounting the logical volume to the application terminal, the verification of the logical bare volume is realized, that is, there is no need to format the logical bare volume before verification, which simplifies the verification process of the logical volume, improves the verification efficiency, realizes the simulation of various real businesses and real business load conditions, and saves data verification time through automatic configuration of the application business model and automatic mounting of the logical volume.
[0035] It should be noted that this solution can perform data verification on multiple logical bare volumes at the same time, that is, it can verify multiple application business models at the same time and support verification in a distributed deployment mode of application terminals.
[0036] Specifically, application business models include but are not limited to web file servers (Web File Server), exchange servers (Exchange Server), media streaming (Media Streaming), online analytical processing (Online Analytical Processing, referred to as OLAP), and online transaction processing (Online Transaction Processing, referred to as OLTP). More specific application business models include monitoring business models, insurance business models, banking business models and medical business models. Application business models are used to simulate real businesses and the load conditions of the above real businesses. For example, the banking business model is used to simulate real banking businesses. The load conditions of real banking businesses are deposit business, loan business and wealth management business, etc.
[0037] Specifically, multiple logical volumes are created in the above-mentioned distributed storage system for the above-mentioned application business models, that is, according to the data volume of the application business model, a corresponding number of logical volumes are configured for the application business model. For example, two logical volumes are configured for the monitoring business model, three logical volumes are configured for the banking business model, and three logical volumes are configured for the medical business model. Of course, the number of logical volumes can be adjusted according to the capacity of each logical volume.
[0038] Specifically, the application terminals include but are not limited to desktop computers and notebook computers.
[0039] Specifically, the multiple logical volumes are automatically mounted to the application terminal using the Iscsi / FC protocol. The logical volumes carry data in the application business model, which is convenient for verification by a data verification tool.
[0040] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0041] In one embodiment of the present application, the data verification tool is applied to verify the reliability of the application business model when it is running in the distributed storage system, including: applying the data verification tool to verify the consistency of write data and read data, the write data is the data in the application business model written in the distributed storage system, and the read data is the data written to the application terminal after the write data is read from the distributed storage system; when the write data and the read data are consistent, it is determined that the application business model is reliable when it is running in the distributed storage system; when the write data and the read data are inconsistent, it is determined that the application business model is unreliable when it is running in the distributed storage system, that is, the reliability of the application business model when it is running in the distributed storage system is verified by verifying the consistency of the write data and the read data.
[0042] Another embodiment of the present application uses the above-mentioned data verification tool to verify the consistency of written data and read data, including: using the above-mentioned data verification tool to process the above-mentioned written data in blocks to obtain multiple first verification data structures; simulating hardware failure; in the process of simulating hardware failure, obtaining the above-mentioned read data; using the above-mentioned data verification tool to process the above-mentioned read data in blocks to obtain multiple second verification data structures; verifying the consistency of the above-mentioned first verification data structure and the corresponding second verification data structure. Specifically, for example, the write data is written from the I / O port each time with a size of 512K data, and the data verification tool processes the 512K data written each time in blocks, for example, into 10 parts, each of which is called a first verification data structure; each time 512K is read from the I / O port For data of size K, the data verification tool divides the 512K data read out each time into blocks, for example, into 10 parts, each of which is called the second verification data structure. By verifying the consistency of the first verification data structure and the second verification data structure, the consistency of the written data and the read data can be obtained in the presence of a hardware failure, so as to realize the consistency verification of the test data. Since hardware failure may cause data loss, that is, cause inconsistency between the written data and the read data, by simulating hardware failures, it can be confirmed in which types of hardware failures the written data and the read data are consistent, and in which types of hardware failures the written data and the read data are inconsistent, so as to adopt corresponding measures to avoid data inconsistency and ensure the reliability of the distributed storage system during actual operation.
[0043] Specifically, each data verification tool has a unique configuration file method, that is, the above-mentioned method of processing write data and read data in blocks is only an enumeration method, and write data and read data can also be processed in other ways as long as identification information is generated to represent the write information and read information.
[0044] Specifically, the above hardware failure includes but is not limited to failure of one or several hardware devices in the distributed storage system. The hardware device failure may be a hard disk failure, a mainboard failure, a graphics card failure, a power supply failure, etc.
[0045] In another embodiment of the present application, after obtaining multiple first verification data structures, the above method also includes storing the multiple first verification data structures, which can be stored in a memory or in a hard disk, to facilitate subsequent comparison with the second data structure.
[0046] In a specific embodiment of the present application, verifying the consistency of the above-mentioned first verification data structure and the corresponding above-mentioned second verification data structure includes: obtaining the first verification part of each of the above-mentioned first verification data structures; obtaining the second verification part of each of the above-mentioned second verification data structures; verifying the consistency of the above-mentioned first verification part and the above-mentioned second verification part, that is, the first verification part is a flag bit of the first verification data structure, the second verification part is a flag bit of the second verification data structure, the first verification part uniquely represents the first verification data structure, and the second verification part uniquely represents the second verification data structure. By comparing the consistency of the first verification part and the second verification part, rapid confirmation of whether the written data and the read data are consistent is achieved.
[0047] In another embodiment of the present application, when the above-mentioned write data and the above-mentioned read data are consistent, determining that the above-mentioned application business model is reliable when running in the above-mentioned distributed storage system includes: when the above-mentioned first verification unit is consistent with the above-mentioned second verification unit, determining that the above-mentioned application business model is reliable when running in the above-mentioned distributed storage system, by determining the reliability of the above-mentioned application business model when running in the above-mentioned distributed storage system, the reliability of the real business when running in the distributed storage system is indirectly determined.
[0048] In another embodiment of the present application, in the case where the above-mentioned write data and the above-mentioned read data are inconsistent, after determining that the above-mentioned application business model is unreliable when running in the above-mentioned distributed storage system, the above-mentioned verification method also includes: the above-mentioned data verification tool exits and generates a consistency problem log at the same time; determines the problematic logical address in the above-mentioned logical volume according to the above-mentioned consistency problem log; determines the software defect in the above-mentioned distributed storage system according to the above-mentioned logical address. In the case where the write data and the above-mentioned read data are inconsistent, there can be a clear consistency problem log, which is convenient for subsequent R&D personnel to locate the specific problematic logical address in the logical volume to determine the software defect, that is, determine the specific location where the software program has a problem, further improving the verification efficiency of the test data. In the prior art solution, there is no obvious log information when a verification consistency problem occurs, and there are obstacles to locating the problem.
[0049] In another embodiment of the present application, the above method further includes recording each specific operation in the logical address in a consistency problem log in the distributed storage system, so as to facilitate taking effective remedial measures after locating the software defect later.
[0050] In a specific embodiment, configuring an application business model includes: applying a python script to automatically configure the above application business model, thereby realizing fast and automatic configuration of multiple application business models. In the prior art, since it is impossible to automatically mount logical volumes and directly mount logical bare volumes, it is impossible to simulate the load conditions of multiple real businesses and the above real businesses. Since this solution can simulate more real application business scenarios, it does not need to consume human and material resources to build real business scenarios, thus saving costs.
[0051] In another specific embodiment, multiple logical volumes are automatically mounted to the application terminal, including: applying a shell or python script to automatically mount multiple logical volumes to the application terminal. Compared with the prior art that cannot automatically mount logical volumes and cannot directly mount logical bare volumes, the verification efficiency of test data is greatly improved.
[0052] The embodiment of the present application also provides a test data verification device. It should be noted that the test data verification device of the embodiment of the present application can be used to execute the test data verification method provided by the embodiment of the present application. The test data verification device provided by the embodiment of the present application is introduced below.
[0053] Figure 2 Schematic diagram of a test data verification device according to an embodiment of the present application. Figure 2 As shown, the device comprises:
[0054] A configuration unit 10 is used to configure an application business model, where the application business model is used to simulate real business and the load of the real business;
[0055] A writing unit 20, used to write the above application business model into the distributed storage system;
[0056] A creation unit 30, configured to create a plurality of logical volumes in the distributed storage system for the application business model;
[0057] The mounting unit 40 is used to automatically mount the plurality of the above-mentioned logical volumes to the application terminal;
[0058] A first generating unit 50, used to generate a data verification tool in the above-mentioned application terminal;
[0059] The verification unit 60 is used to use the data verification tool to verify the reliability of the application business model when it runs in the distributed storage system.
[0060] In the above scheme, the creation unit creates a logical volume for the corresponding application business model in the distributed storage system, the mounting unit automatically mounts the created logical volume to the application terminal, the first generation unit generates a data verification tool in the application terminal, and the verification unit uses the data verification tool to verify the reliability of the application business model when running in the distributed storage system. By automatically mounting the logical volume to the application terminal, the verification of the logical bare volume is realized, that is, there is no need to format the logical bare volume before verification, which simplifies the verification process of the logical volume, improves the verification efficiency, realizes the simulation of multiple real businesses and real business load conditions, and saves data verification time through automatic configuration of the application business model and automatic mounting of the logical volume.
[0061] It should be noted that this solution can perform data verification on multiple logical bare volumes at the same time, that is, it can verify multiple application business models at the same time and support verification in a distributed deployment mode of application terminals.
[0062] Specifically, application business models include but are not limited to web file servers (Web File Server), exchange servers (Exchange Server), media streaming (Media Streaming), online analytical processing (Online Analytical Processing, referred to as OLAP), and online transaction processing (Online Transaction Processing, referred to as OLTP). More specific application business models include monitoring business models, insurance business models, banking business models and medical business models. Application business models are used to simulate real businesses and the load conditions of the above real businesses. For example, the banking business model is used to simulate real banking businesses. The load conditions of real banking businesses are deposit business, loan business and wealth management business, etc.
[0063] Specifically, multiple logical volumes are created in the above-mentioned distributed storage system for the above-mentioned application business models, that is, according to the data volume of the application business model, a corresponding number of logical volumes are configured for the application business model. For example, two logical volumes are configured for the monitoring business model, three logical volumes are configured for the banking business model, and three logical volumes are configured for the medical business model. Of course, the number of logical volumes can be adjusted according to the capacity of each logical volume.
[0064] Specifically, the application terminals include but are not limited to desktop computers and notebook computers.
[0065] Specifically, the multiple logical volumes are automatically mounted to the application terminal using the Iscsi / FC protocol. The logical volumes carry data in the application business model, which is convenient for verification by a data verification tool.
[0066] In one embodiment of the present application, the verification unit includes a verification module, a first determination module and a second determination module, the verification module is used to apply the above-mentioned data verification tool to verify the consistency of the written data and the read data, the above-mentioned written data is the data written into the above-mentioned application business model in the above-mentioned distributed storage system, and the above-mentioned read data is the data written to the above-mentioned application terminal after the above-mentioned written data is read from the above-mentioned distributed storage system; the first determination module is used to determine that the above-mentioned application business model is reliable when running in the above-mentioned distributed storage system when the above-mentioned written data and the above-mentioned read data are consistent; the second determination module is used to determine that the above-mentioned application business model is unreliable when running in the above-mentioned distributed storage system when the above-mentioned written data and the above-mentioned read data are inconsistent, that is, the reliability of the application business model when running in the above-mentioned distributed storage system is verified by verifying the consistency of the written data and the read data.
[0067] In another embodiment of the present application, the verification module includes a first block submodule, a simulation submodule, an acquisition submodule, a second block submodule and a verification submodule, the first block submodule is used to apply the above-mentioned data verification tool to block the above-mentioned write data to obtain multiple first verification data structures; the simulation submodule is used to simulate hardware failure; the acquisition submodule is used to obtain the above-mentioned read data in the process of simulating hardware failure; the second block submodule is used to apply the above-mentioned data verification tool to block the above-mentioned read data to obtain multiple second verification data structures; the verification submodule is used to verify the consistency of the above-mentioned first verification data structure and the corresponding second verification data structure. Specifically, for example, the write data is written from the I / O port each time 512K of data, and the data verification tool blocks the 512K data written each time, for example, into 10 parts, each part of data It is called the first verification data structure; 512K of data is read out from the I / O port each time, and the data verification tool divides the 512K data read out each time into blocks, for example, into 10 parts, each of which is called the second verification data structure. By verifying the consistency of the first verification data structure and the second verification data structure, it is possible to obtain the consistency of the written data and the read data in the presence of a hardware failure, so as to achieve consistency verification of the test data. Since a hardware failure may cause data loss, that is, cause inconsistency between the written data and the read data, it is possible to confirm under what types of hardware failures the written data and the read data are consistent, and under what types of hardware failures the written data and the read data are inconsistent, by simulating hardware failures, so as to adopt corresponding measures to avoid data inconsistency and ensure the reliability of the distributed storage system during actual operation.
[0068] Specifically, each data verification tool has a unique configuration file method, that is, the above-mentioned method of processing write data and read data in blocks is only an enumeration method, and write data and read data can also be processed in other ways as long as identification information is generated to represent the write information and read information.
[0069] Specifically, the above hardware failure includes but is not limited to failure of one or several hardware devices in the distributed storage system. The hardware device failure may be a hard disk failure, a mainboard failure, a graphics card failure, a power supply failure, etc.
[0070] In another embodiment of the present application, the above-mentioned device also includes a storage unit, which is used to store the multiple first verification data structures after obtaining the multiple first verification data structures. The storage unit can be stored in a memory or a hard disk to facilitate subsequent comparison with the second data structure.
[0071] In a specific embodiment of the present application, the check submodule is also used to obtain the first check part of each of the above-mentioned first check data structures; obtain the second check part of each of the above-mentioned second check data structures; and check the consistency of the above-mentioned first check part and the above-mentioned second check part, that is, the first check part is a flag bit of the first check data structure, and the second check part is a flag bit of the second check data structure. The first check part uniquely represents the first check data structure, and the second check part uniquely represents the second check data structure. By comparing the consistency of the first check part and the second check part, a quick confirmation of whether the written data and the read data are consistent is achieved.
[0072] In another embodiment of the present application, the first determination module is also used to determine that the application business model is reliable when running in the distributed storage system when the first verification unit is consistent with the second verification unit. By determining the reliability of the application business model when running in the distributed storage system, the reliability of the real business when running in the distributed storage system is indirectly determined.
[0073] In another embodiment of the present application, the verification device further includes a second generation unit, a first determination unit and a second determination unit, wherein the second generation unit is used for, when the write data and the read data are inconsistent, determining that the application business model is unreliable when running in the distributed storage system, and then the data verification tool exits and generates a consistency problem log at the same time; the first determination unit is used for, when the write data and the read data are inconsistent, determining that the application business model is unreliable when running in the distributed storage system, and then determining the problematic logical address in the logical volume according to the consistency problem log; the second determination unit is used for, when the write data and the read data are inconsistent, determining that the application business model is unreliable when running in the distributed storage system, and then determining the software defect in the distributed storage system according to the logical address, in the case of inconsistency between the write data and the read data, there can be a clear consistency problem log, which is convenient for subsequent R&D personnel to locate the problematic specific logical address in the logical volume to determine the software defect, that is, determine the specific location of the software program with the problem, and further improve the verification efficiency of the test data, while the prior art solution has no obvious log information when verification consistency problems occur, and there are obstacles to locating the problem.
[0074] In another embodiment of the present application, the above-mentioned device also includes recording each specific operation in the logical address in a consistency problem log in the distributed storage system, so as to take effective remedial measures after locating the software defects later.
[0075] In a specific embodiment, configuring an application business model includes: applying a python script to automatically configure the above application business model, thereby realizing fast and automatic configuration of multiple application business models. In the prior art, since it is impossible to automatically mount logical volumes and directly mount logical bare volumes, it is impossible to simulate the load conditions of multiple real businesses and the above real businesses. Since this solution can simulate more real application business scenarios, it does not need to consume human and material resources to build real business scenarios, thus saving costs.
[0076] In another specific embodiment, multiple logical volumes are automatically mounted to the application terminal, including: applying a shell or python script to automatically mount multiple logical volumes to the application terminal. Compared with the prior art that cannot automatically mount logical volumes and cannot directly mount logical bare volumes, the verification efficiency of test data is greatly improved.
[0077] The above-mentioned test data verification device includes a processor and a memory. The above-mentioned configuration unit, writing unit, creation unit, mounting unit, first generation unit and verification unit are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize corresponding functions.
[0078] The processor includes a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be set, and the verification efficiency of the test data can be improved by adjusting the kernel parameters.
[0079] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.
[0080] An embodiment of the present invention provides a storage medium on which a program is stored. When the program is executed by a processor, the above-mentioned test data verification method is implemented.
[0081] An embodiment of the present invention provides a processor, and the processor is used to run a program, wherein the verification method of the test data is executed when the program is running.
[0082] An embodiment of the present invention provides a device, the device comprising a processor, a memory, and a program stored in the memory and executable on the processor, and when the processor executes the program, at least the following steps are implemented:
[0083] Step S101, configuring an application business model, where the application business model is used to simulate real business and the load of the real business;
[0084] Step S102, writing the above application business model into the distributed storage system;
[0085] Step S103, creating multiple logical volumes in the distributed storage system for the application business model;
[0086] Step S104, automatically mounting the plurality of the above logical volumes to the application terminal;
[0087] Step S105, generating a data verification tool in the above application terminal;
[0088] Step S106: Use the data verification tool to verify the reliability of the application business model when it runs in the distributed storage system.
[0089] The devices in this article can be servers, PCs, PADs, mobile phones, etc.
[0090] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program for initializing at least the following method steps:
[0091] Step S101, configuring an application business model, where the application business model is used to simulate real business and the load of the real business;
[0092] Step S102, writing the above application business model into the distributed storage system;
[0093] Step S103, creating multiple logical volumes in the distributed storage system for the application business model;
[0094] Step S104, automatically mounting the plurality of the above logical volumes to the application terminal;
[0095] Step S105, generating a data verification tool in the above application terminal;
[0096] Step S106: Use the data verification tool to verify the reliability of the application business model when it runs in the distributed storage system.
[0097] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.
[0098] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0099] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0100] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0101] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0102] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0103] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0104] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0105] From the above description, it can be seen that the above embodiments of the present application achieve the following technical effects:
[0106] 1) The test data verification method of the present application creates a logical volume for the corresponding application business model in a distributed storage system, and then automatically mounts the created logical volume to an application terminal, generates a data verification tool in the application terminal, and uses the data verification tool to verify the reliability of the application business model when it is running in the distributed storage system. By automatically mounting the logical volume to the application terminal, the verification of the logical bare volume is realized, that is, there is no need to format the logical bare volume before verification, which simplifies the verification process of the logical volume, improves the verification efficiency, realizes the simulation of multiple real businesses and real business load conditions, and saves data verification time through automatic configuration of the application business model and automatic mounting of the logical volume.
[0107] 2) The test data verification device of the present application, the creation unit creates a logical volume for the corresponding application business model in the distributed storage system, the mounting unit automatically mounts the created logical volume to the application terminal, the first generation unit generates a data verification tool in the application terminal, and the verification unit uses the data verification tool to verify the reliability of the application business model when running in the distributed storage system. By automatically mounting the logical volume to the application terminal, the verification of the logical bare volume is realized, that is, there is no need to format the logical bare volume before verification, which simplifies the verification process of the logical volume, improves the verification efficiency, realizes the simulation of multiple real businesses and real business load conditions, and saves data verification time through automatic configuration of the application business model and automatic mounting of the logical volume.
[0108] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims
1. A method for verifying test data, characterized in that: include: Configuring an application business model, wherein the application business model is used to simulate real business and the load condition of the real business; Writing the application business model into a distributed storage system; Creating a plurality of logical volumes in the distributed storage system for the application business model; Automatically mounting the plurality of logical volumes to an application terminal; Generating a data verification tool in the application terminal; Using the data verification tool to verify the reliability of the application business model when running in the distributed storage system, Applying the data verification tool to verify the reliability of the application business model when running in the distributed storage system includes: applying the data verification tool to verify the consistency of write data and read data, the write data is data in the application business model written in the distributed storage system, and the read data is data written to the application terminal after the write data is read from the distributed storage system; when the write data and the read data are consistent, determining that the application business model is reliable when running in the distributed storage system; when the write data and the read data are inconsistent, determining that the application business model is unreliable when running in the distributed storage system, In the case where the written data and the read data are inconsistent, after determining that the application business model is unreliable when running in the distributed storage system, the verification method also includes: the data verification tool exits and generates a consistency problem log; determines the problematic logical address in the logical volume according to the consistency problem log; and determines the software defect in the distributed storage system according to the logical address.
2. The verification method according to claim 1, characterized in that: Applying the data verification tool to verify the consistency of written data and read data includes: Applying the data verification tool to process the written data in blocks to obtain a plurality of first verification data structures; Simulate hardware failures; In the process of simulating hardware failure, obtaining the read data; Applying the data verification tool to process the read data in blocks to obtain a plurality of second verification data structures; Verify the consistency of the first verification data structure and the corresponding second verification data structure.
3. The verification method according to claim 2, characterized in that: Verifying the consistency of the first verification data structure and the corresponding second verification data structure includes: Obtaining a first verification part of each of the first verification data structures; Obtaining a second verification part of each of the second verification data structures; The consistency between the first verification unit and the second verification unit is verified.
4. The verification method according to claim 3, characterized in that: When the written data and the read data are consistent, determining that the application business model is reliable when running in the distributed storage system includes: When the first verification unit is consistent with the second verification unit, it is determined that the application business model is reliable when running in the distributed storage system.
5. The verification method according to any one of claims 1 to 4, characterized in that: Configure the application business model, including: The application business model is automatically configured using a python script.
6. The verification method according to any one of claims 1 to 4, characterized in that: Automatically mounting the plurality of logical volumes to the application terminal comprises: Using a shell or python script, the plurality of logical volumes are automatically mounted to the application terminal.
7. A test data verification device, characterized in that: include: A configuration unit, used to configure an application business model, wherein the application business model is used to simulate a real business and a load condition of the real business; A writing unit, used for writing the application business model into a distributed storage system; A creating unit, configured to create a plurality of logical volumes in the distributed storage system for the application business model; A mounting unit, used for automatically mounting the plurality of logical volumes to an application terminal; A first generating unit, used to generate a data verification tool in the application terminal; A verification unit, used to use the data verification tool to verify the reliability of the application business model when it is running in the distributed storage system, The verification unit includes a verification module, a first determination module and a second determination module. The verification module is used to apply the above-mentioned data verification tool to verify the consistency of the written data and the read data. The above-mentioned written data is the data in the above-mentioned application business model written in the above-mentioned distributed storage system, and the above-mentioned read data is the data written to the above-mentioned application terminal after the above-mentioned written data is read from the above-mentioned distributed storage system; the first determination module is used to determine that the above-mentioned application business model is reliable when running in the above-mentioned distributed storage system when the above-mentioned written data and the above-mentioned read data are consistent; the second determination module is used to determine that the above-mentioned application business model is unreliable when running in the above-mentioned distributed storage system when the above-mentioned written data and the above-mentioned read data are inconsistent, that is, the reliability of the application business model when running in the above-mentioned distributed storage system is verified by verifying the consistency of the written data and the read data. In the case where the written data and the read data are inconsistent, after determining that the application business model is unreliable when running in the distributed storage system, the device also includes: a first generation unit, used for the data verification tool to exit and generate a consistency problem log; a first determination unit is used to determine the problematic logical address in the logical volume according to the consistency problem log; and a second determination unit is used to determine the software defect in the distributed storage system according to the logical address.
8. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the test data verification method according to any one of claims 1 to 6.
9. A processor, characterized in that: The processor is used to run a program, wherein the program executes the test data verification method according to any one of claims 1 to 6 when running.
