Data access control method and device, computer equipment and storage medium

Abstract

The invention relates to the technical field of storage, and discloses a data access control method and device, computer equipment and a storage medium, and the data access control method comprises the following steps: migrating source volume data to a target volume based on data fragments in a source volume; mapping information corresponding to the data fragments is obtained, and the mapping information is used for indicating the access state of the data fragments; after a processing request for target data in the data fragments is detected, obtaining an access state of the target data based on the mapping information; and when the access state meets the execution condition of the processing request, processing the target data. According to the method and the device, the target data can be prevented from being migrated and other processing when the target data is in the access state such as the write-in data which does not meet the execution condition, so that the consistency of the data in the source volume and the target volume is kept, and the data loss in the data migration process is reduced.

Description

Technical Field

[0001] The present invention relates to the field of storage technology, and in particular to a data access control method, device, computer equipment and storage medium. Background Art

[0002] When technology upgrades, business requirements change, companies merge, or data center migrations occur, it is often necessary to move data from one storage system or database to another through data migration.

[0003] However, in related data migration solutions, when the storage LUN (Logical Unit Number) back-end storage uses distributed block devices to perform online LUN data migration, new data may be written during the data migration process, or the migrated data may be modified, resulting in inconsistency between the source volume and the migration target volume, thereby causing data loss. Summary of the invention

[0004] In view of this, the present invention provides a data access control method, apparatus, computer device and storage medium to solve the problem of data inconsistency between a source volume and a migration target volume due to request conflicts during data migration.

[0005] In a first aspect, the present invention provides a data access control method, the method comprising:

[0006] Migrate source volume data to the target volume based on data shards in the source volume;

[0007] Obtaining mapping information corresponding to the data shard, wherein the mapping information is used to indicate the access status of the data shard;

[0008] After detecting a processing request for target data in the data shard, obtaining an access status of the target data based on the mapping information;

[0009] When the access status satisfies the execution condition of the processing request, the target data is processed.

[0010] In an optional implementation, migrating source volume data to a target volume based on data slices in the source volume includes:

[0011] After detecting the data migration instruction, determining the source storage pool and the target storage pool indicated by the data migration instruction, and determining the source volume according to the source storage pool;

[0012] Read the attribute information of the source volume;

[0013] Based on the attribute information, create a target volume corresponding to the source volume in the target storage pool;

[0014] Obtain the historical load data and volume capacity corresponding to the source volume;

[0015] Calculate the load average corresponding to the source volume based on the historical load data volume;

[0016] Determine the data capacity of the data shard based on a preset functional relationship, a load average and a volume capacity, wherein in the preset functional relationship, the data capacity of the data shard is inversely proportional to the load average, and the data capacity of the data shard is directly proportional to the volume capacity;

[0017] Determine the number of shards of the data shard based on the ratio of the volume capacity to the data capacity of the data shard;

[0018] Generate mapping information based on the number of shards;

[0019] Based on data sharding, perform data migration operations on the source volume data.

[0020] In an embodiment of the present invention, before performing data migration, the data in the source volume may be firstly sharded to obtain data shards. When determining the data shards, the data shards may not be affected by the original data shards in the source volume, but the data shards are calculated by the load average and the volume capacity, thereby reducing the impact on the business while improving the efficiency of data migration.

[0021] In an optional implementation, the preset functional relationship includes: an inverse proportional function;

[0022] Based on the preset function relationship, load average and volume capacity, the data capacity of the data shard is determined, including:

[0023] Get the preset load constant;

[0024] Determine the independent variable of the inverse proportional function based on the sum of the load constant and the load mean;

[0025] The data capacity of the data slice is determined based on the product of the volume capacity and the inverse proportional function.

[0026] In the embodiment of the present invention, the data capacity of the data shard is inversely proportional to the load average, and the data capacity of the data shard is directly proportional to the volume capacity. Therefore, when the volume carries more services and the load average L is large, the data shard can be reduced to reduce the data range accessed each time during the migration and data verification process, thereby reducing the impact on the service. In addition, when the volume capacity is large, the data shard can be enlarged to increase the data range accessed each time during the data migration and data verification process to improve the efficiency of data migration.

[0027] In an optional implementation, the access state includes: a first state, a second state, and a third state, wherein the first state is used to indicate that the target data is being migrated to the target volume, the second state is used to indicate that data is being written to the data slice corresponding to the target data, and the third state is used to indicate that the data slice corresponding to the target data is in a stateless state;

[0028] When the access status meets the execution conditions of the processing request, the target data is processed, including:

[0029] determining whether the access state is a third state;

[0030] If not, repeatedly monitor the access status corresponding to the target data until the access status is a third state;

[0031] If so, it is determined that the access status satisfies the execution condition, and the target data is processed in response to the processing request, and the access status corresponding to the target data in the mapping information is modified when executing the processing request.

[0032] In an embodiment of the present invention, before executing a processing request, it is possible to determine whether the access status of the target data is empty, and if it is not empty, wait until the access status is empty before executing the processing request, thereby avoiding processing request conflicts and maintaining the consistency of data in the source volume and the target volume.

[0033] In an optional embodiment, the processing request includes: a first request, a second request and a third request, wherein the first request is used to migrate the target data to the target volume, the second request is used to write data in the data slice corresponding to the target data, and the third request is used to read the target data.

[0034] In an embodiment of the present invention, when executing a read request for target data, it can be detected whether the task queue is empty. If so, the third request is directly executed to read the target data in the source volume; if not, it means that a processing request for the target data is being executed at this time, and the third request can be added to the end position in the task queue to ensure that the target data is read after the processing request for the target data is completed, thereby ensuring the accuracy of the read target data.

[0035] In an optional implementation, when the access status satisfies the execution condition of the processing request, processing the target data further includes:

[0036] When the processing request is the third request, obtaining a task queue and determining whether the task queue is empty, wherein the task queue is used to cache processing requests for target data;

[0037] If not, cache the third request to the end of the task queue;

[0038] If so, in response to the third request, the target data is read.

[0039] In an embodiment of the present invention, when executing a read request for target data, it can be detected whether the task queue is empty. If so, the third request is directly executed to read the target data in the source volume; if not, it means that a processing request for the target data is being executed at this time, and the third request can be added to the end position in the task queue to ensure that the target data is read after the processing request for the target data is completed, thereby ensuring the accuracy of the read target data.

[0040] In an optional embodiment, the method further includes:

[0041] After detecting a fourth request for the target data, obtaining a migration progress for the source volume data, wherein the fourth request is used to instruct to verify the target data;

[0042] When the migration progress meets the verification conditions, the target data is verified, and when the verification result meets the alarm conditions, an alarm message is generated.

[0043] In an embodiment of the present invention, it is possible to verify whether the migration result meets the alarm condition during the data migration process, without having to perform data verification after the migration is completed. This solves the problem of difficulty in performing data verification during ongoing business processes, ensures the security and reliability of data during the entire migration process, and caters to the current concept of fast storage and efficient storage.

[0044] In an optional implementation, when the migration progress meets the verification condition, verifying the target data includes:

[0045] Get the first shard sequence number of the data shard corresponding to the migration progress;

[0046] When the difference between the first shard sequence number and the second shard sequence number corresponding to the target data exceeds a preset threshold, it is determined that the migration progress meets the verification condition, and the target data is verified.

[0047] In an embodiment of the present invention, data verification is performed on the data shards that have completed the migration during the data migration process. Therefore, in order to avoid conflicts between the verification progress and the data migration progress, the verification progress can lag behind the migration progress, thereby solving the problem of difficulty in performing data verification during continuous business processes, ensuring the security and reliability of data during the entire migration process, and catering to the current concepts of fast storage and efficient storage.

[0048] In a second aspect, the present invention provides a data access control device, the device comprising:

[0049] A migration module, used to migrate source volume data to a target volume based on data slices in the source volume;

[0050] A first acquisition module, used to acquire mapping information corresponding to the data shard, wherein the mapping information is used to indicate the access status of the data shard;

[0051] A second acquisition module is used to acquire the access status of the target data based on the mapping information after detecting a processing request for the target data in the data shard;

[0052] The processing module is used to process the target data when the access status meets the execution condition of the processing request.

[0053] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the data access control method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0054] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the data access control method of the first aspect or any corresponding embodiment thereof.

[0055] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions for causing a computer to execute the data access control method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0057] Figure 1 is a flow chart of a data access control method according to an embodiment of the present invention;

[0058] Figure 2 is a flow chart of another data access control method according to an embodiment of the present invention;

[0059] Figure 3 is a schematic diagram of a data verification process according to an embodiment of the present invention;

[0060] Figure 4 is a structural block diagram of a data access control device according to an embodiment of the present invention;

[0061] Figure 5 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0063] In conjunction with the application scenarios on which the execution of the data access control method depends, the application scenarios are described here.

[0064] When technology upgrades, business requirements change, companies merge, or data center migrations occur, it is often necessary to move data from one storage system or database to another through data migration.

[0065] However, in related data migration solutions, when the storage LUN (Logical Unit Number) back-end storage uses distributed block devices to perform online LUN data migration, new data may be written during the data migration process, or the migrated data may be modified, resulting in inconsistency between the source volume and the migration target volume, thereby causing data loss.

[0066] Based on this, an embodiment of the present invention provides a data access control method, which can first migrate the source volume data to the target volume based on the data slices in the source volume, and obtain the mapping information corresponding to the data slices, wherein the mapping information is used to indicate the access status of the data slices. Next, after detecting a processing request for the target data in the data slices, the access status of the target data can be obtained based on the mapping information. Then, when the access status meets the execution condition of the processing request, the target data can be processed, thereby avoiding the migration of the target data when the target data is in an access state that does not meet the execution condition, such as writing data, so as to maintain the consistency of the data in the source volume and the target volume, and reduce data loss during the data migration process.

[0067] According to an embodiment of the present invention, a data access control method embodiment is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0068] In this embodiment, a data access control method is provided, which can be used for various storage devices and computing devices with data storage and processing functions, such as hard disk drives, solid-state drives, hybrid storage, and cloud storage services. Figure 1 is a flow chart of a data access control method according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0069] Step S101, based on the data slices in the source volume, migrate the source volume data to the target volume.

[0070] In an embodiment of the present invention, both the source volume and the target volume can be developed in C++ language, in the block storage layer of the distributed storage system, wherein the source volume is the starting location of data migration, and the target volume refers to the final destination of the data. For example, the source volume can be a hard disk drive, a solid-state drive, a network attached storage (NAS) or other forms of storage devices. The target volume is the recipient of the data migration, which may be a new storage device or another location, such as a new server, cloud infrastructure or other types of storage systems.

[0071] When technology upgrades, business demand changes, corporate mergers, or data center migrations occur, users can initiate a data migration task. When the data migration task is executed, the data in the source volume can be migrated to the target volume in sequence according to the data shards. The data in the source volume can be divided into multiple data shards in advance to improve the efficiency of data migration and the scalability of the system.

[0072] Step S102, obtaining mapping information corresponding to the data slice, wherein the mapping information is used to indicate the access status of the data slice.

[0073] Step S103, after detecting a processing request for target data in the data slice, obtaining the access status of the target data based on the mapping information.

[0074] In an embodiment of the present invention, a processing request may include a data migration request and a write IO request. Corresponding to the processing request, the mapping information may include a bitmap, and the bitmap may include an access status corresponding to each data shard. Here, the access status includes a data migration status M, a business access status I, and no status, etc., wherein the data migration status is used to indicate that the data in the source volume data shard is migrated to the target volume in response to the data migration request, the business access status may be used to indicate that a write IO request is initiated for the data shard, and the no status is used to indicate that the data in the data shard is not migrated or accessed.

[0075] For example, if the access status read from the data slice at the i-th position in the bitmap is M, it means that the data slice at the i-th position is in the data migration state. If the access status read from the data slice at the b-th position in the bitmap is empty, it means that the data slice at the b-th position is in the unaccessed state.

[0076] It should be understood that during the data migration process, the bitmap can be updated in real time according to the change of the access state corresponding to the data slice. For example, after the data migration in the above-mentioned i-th data slice is completed, the access state of the data slice can be changed from M to stateless.

[0077] Step S104: When the access status satisfies the execution condition of the processing request, the target data is processed.

[0078] In an embodiment of the present invention, considering that simultaneously initiating multiple processing requests for target data for the same data shard may cause conflicts and lead to inconsistent data before and after migration, when a processing request is monitored, the execution conditions pre-set for the processing request can be obtained.

[0079] For example, if the processing request is a data migration request, the execution condition set for the data migration request may be that the access state is not I; for another example, if the processing request is a write IO request, the execution condition set for the data migration request may be that the access state is not M, thereby avoiding data conflicts caused by executing the data migration request and the write IO request on the target data at the same time.

[0080] It can be known from the above description that in an embodiment of the present invention, first, based on the data slices in the source volume, the source volume data can be migrated to the target volume, and the mapping information corresponding to the data slices can be obtained, wherein the mapping information is used to indicate the access status of the data slices. Next, after detecting a processing request for the target data in the data slices, the access status of the target data can be obtained based on the mapping information. Then, when the access status meets the execution condition of the processing request, the target data can be processed, thereby avoiding the migration and other processing of the target data when the target data is in an access state that does not meet the execution condition, such as writing data, so as to maintain the consistency of the data in the source volume and the target volume, and reduce data loss during the data migration process.

[0081] In this embodiment, another data access control method is provided, which can be used for various storage devices and computing devices with data storage and processing functions, such as hard disk drives, solid-state drives, hybrid storage, and cloud storage services. Figure 2 is a flow chart of another data access control method according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0082] Step S201: Migrate source volume data to a target volume based on data slices in the source volume.

[0083] Specifically, the above step S201 includes:

[0084] Step S2011, after detecting the data migration instruction, determining the source storage pool and the target storage pool indicated by the data migration instruction, and determining the source volume according to the source storage pool.

[0085] Step S2012, read the attribute information of the source volume.

[0086] Step S2013: Create a target volume corresponding to the source volume in the target storage pool based on the attribute information.

[0087] Step S2014, obtaining the historical load data volume and volume capacity corresponding to the source volume.

[0088] Step S2015: Calculate the load mean corresponding to the source volume based on the historical load data volume.

[0089] Step S2016, based on a preset functional relationship, load average and volume capacity, determine the data capacity of the data shard. In the preset functional relationship, the data capacity of the data shard is inversely proportional to the load average, and the data capacity of the data shard is directly proportional to the volume capacity.

[0090] Step S2017, determining the number of shards of the data shard based on the ratio of the volume capacity to the data capacity of the data shard.

[0091] Step S2018, generating mapping information according to the number of shards.

[0092] Step S2019: performing data migration operations on the source volume data based on the data sharding.

[0093] In the embodiment of the present invention, the source storage pool generally refers to a large storage area formed by combining multiple hard disks or storage devices through a specific logic. Here, these hard disks may be combined together through RAID (Redundant Array of Independent Disks) technology to improve data reliability and read and write performance. In contrast, the source volume refers to an independent storage space divided on the basis of the storage pool. The user can create multiple volumes in a storage pool, and each volume can be regarded as an independent hard disk for formatting, partitioning and other operations. The target storage pool is a storage area for storing migrated data in the source storage pool.

[0094] When creating a target volume in the target storage pool, you can create a target volume with the same attribute information as the source volume, thereby improving the success rate of the data migration operation. The attribute information may include special attributes such as identifier, file system type, size, status, snapshot, etc.

[0095] Before performing data migration, the data capacity size S of the data shard can be calculated. Specifically, the data capacity size S can be calculated based on the load mean L corresponding to the source volume and the volume capacity C, wherein the load mean can be the mean of the historical load data volume of the source volume within a preset historical period, thereby indicating the load amount of the source volume within the preset period.

[0096] It should be understood that the data capacity S of the data shard can be determined based on the preset functional relationship, the load mean L and the volume capacity C, wherein in the preset functional relationship, the data capacity S of the data shard is inversely proportional to the load mean L and is directly proportional to the volume capacity C. In other words, when the load mean L of the source volume is large, it means that the volume carries more services, and the data shard should be reduced to reduce the data range accessed each time during the migration and data verification process, thereby reducing the impact on the service. In addition, when the volume capacity C is large, the data shard should be enlarged to increase the data range accessed each time during the data migration and data verification process to improve the efficiency of data migration.

[0097] After determining the data capacity S of the data shard, the number of shards of the data shard may be determined based on C / S to obtain the total number of data shards, so as to establish a bitmap according to the total number, the bitmap including the access status corresponding to each data shard.

[0098] Step S202: Obtain mapping information corresponding to the data shard, where the mapping information is used to indicate the access status of the data shard. Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0099] Step S203: After detecting a processing request for the target data in the data shard, obtain the access status of the target data based on the mapping information. Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0100] Step S204: When the access status satisfies the execution condition of the processing request, the target data is processed. Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0101] In an embodiment of the present invention, before performing data migration, the data in the source volume may be firstly sharded to obtain data shards. When determining the data shards, the data shards may not be affected by the original data shards in the source volume, but the data shards are calculated by the load average and the volume capacity, thereby reducing the impact on the business while improving the efficiency of data migration.

[0102] In some optional implementations, the preset functional relationship includes: an inverse proportional function, and the step S2016 includes:

[0103] Step a1, obtaining a preset load constant.

[0104] Step a2: determining the independent variable of the inverse proportional function based on the sum of the load constant and the load mean.

[0105] Step a3: determining the data capacity of the data slice based on the product of the volume capacity and the inverse proportional function.

[0106] In the embodiment of the present invention, the above inverse proportional function can be expressed as The load constant may be ε, where the independent variable x=L+ε. Then, when the data capacity S of the data slice is calculated by the inverse proportional function,

[0107] It should be understood that in the above inverse proportional function In the data sharding, the data capacity S of the data shard is inversely proportional to the load average L, and the data capacity a of the data shard is directly proportional to the volume capacity C. Therefore, when the volume carries more services and the load average L is large, the data sharding can be reduced to reduce the data range accessed each time during the migration and data verification process, thereby reducing the impact on the services. In addition, when the volume capacity C is large, the data sharding can be enlarged to increase the data range accessed each time during the data migration and data verification process to improve the efficiency of data migration.

[0108] In some optional embodiments, the above-mentioned processing request includes: a first request, a second request and a third request, wherein the first request is used to migrate the target data to the target volume, the second request is used to write data in the data slice corresponding to the target data, and the third request is used to read the target data.

[0109] In the embodiment of the present invention, the first request is the above-mentioned data migration request, the second request is the above-mentioned write IO request, and the third request is a read IO request. Specifically, when the processing request is the third request, the above-mentioned step S104, when the access status satisfies the execution condition of the processing request, processes the target data, including:

[0110] Step S11, when the processing request is the third request, obtain the task queue and determine whether the task queue is empty, wherein the task queue is used to cache processing requests for target data; if not, execute step S12, and if so, execute step S13.

[0111] Step S12: caching the third request to the end of the task queue.

[0112] Step S13, reading target data in response to the third request.

[0113] In an embodiment of the present invention, when the third request is detected, the LBA (Logical Block Address) and data length of the data to be read indicated by the third request can be obtained to calculate the data fragment corresponding to the target data indicated by the third request.

[0114] Specifically, the role of the above-mentioned LBA is to provide a unified and concise address mapping method for the storage device. Through this mapping, the data on the storage device can be easily accessed and operated. The data length of the target data indicated by the third request can be cross-shard, that is, by initiating a third request, the target data in multiple data shards can be read.

[0115] For example, if the data LBA indicated by the third request to be read is the i-th data slice and the data length is 3C, then the target data read in response to the third request may include the data in the i-th data slice, the i+1-th data slice and the i+2-th data slice.

[0116] It should be understood that when executing the above-mentioned third request, it is possible to detect whether the task queue is empty, that is, whether the task queue contains the above-mentioned first request and second request. If so, the third request can be directly executed to read the target data in the source volume; if not, the third request can be added to the end position in the task queue to ensure that the target data is read after the processing request of the target data is executed, thereby ensuring the accuracy of the read target data.

[0117] In an embodiment of the present invention, when executing a read request for target data, it can be detected whether the task queue is empty. If so, the third request is directly executed to read the target data in the source volume; if not, it means that a processing request for the target data is being executed at this time, and the third request can be added to the end position in the task queue to ensure that the target data is read after the processing request for the target data is completed, thereby ensuring the accuracy of the read target data.

[0118] In some optional implementations, the access state includes: a first state, a second state, and a third state, wherein the first state is used to indicate that the target data is being migrated to the target volume, the second state is used to indicate that data is being written to the data slice corresponding to the target data, and the third state is used to indicate that the data slice corresponding to the target data is in a stateless state; the step S104 further includes:

[0119] Step S21, determine whether the access state is the third state; if not, execute step S22, and if so, execute step S23.

[0120] Step S22, repeatedly monitoring the access status corresponding to the target data until the access status is a third status.

[0121] Step S23, determining that the access status satisfies the execution condition, and processing the target data in response to the processing request, and modifying the access status corresponding to the target data in the mapping information when executing the processing request.

[0122] In an embodiment of the present invention, the access status of the data slice corresponding to the target data can be queried based on the bitmap, where the access status includes the first status, the second status and the third status. It should be understood that when the processing request is the first request or the second request, the target data can be directly executed in response to the processing request only when the access status corresponding to the target data is the third status.

[0123] It should be understood that when the processing request is the first request (i.e., the data migration request), the access status corresponding to the target data can be the second state and the third state. At this time, if the access status found is the second state, the first request can be suspended, and the access status of the target data can be repeatedly detected until the access status is empty. At this time, it means that the second request is executed. Similarly, when the processing request is the second request (i.e., the write IO request), the access status corresponding to the target data can be the first state and the third state. At this time, if the access status found is the first state, the second request can be suspended, and the access status of the target data can be repeatedly detected until the access status is empty.

[0124] It should be understood that when executing the above processing request, the access state of the data slice corresponding to the target data in the bitmap can be changed to the access state corresponding to the processing request. Specifically, the access state corresponding to the first request can be the above first state M, and the access state corresponding to the second request can be the above second state I.

[0125] In an embodiment of the present invention, before executing a processing request, it is possible to determine whether the access status of the target data is empty, and if it is not empty, wait until the access status is empty before executing the processing request, thereby avoiding processing request conflicts and maintaining the consistency of data in the source volume and the target volume.

[0126] In some optional embodiments, the above Figure 1 The corresponding implementation method also includes:

[0127] Step S31, after detecting a fourth request for target data, obtaining the migration progress for source volume data, wherein the fourth request is used to instruct to verify the target data.

[0128] Step S32: When the migration progress meets the verification condition, the target data is verified, and when the verification result meets the alarm condition, an alarm message is generated.

[0129] In the embodiment of the present invention, during the data migration process, the migrated data in the target volume can be verified by data verification to ensure the security and reliability of the data during the entire migration process. Here, the verification result can be used to indicate whether the data in the target volume is consistent with the source volume.

[0130] In specific implementation, the data slice corresponding to the target data in the source volume A can be read, and the data MD5 value A_c corresponding to the target data can be calculated. Similarly, the data MD5 value B_c corresponding to the target data in the target volume can be calculated. Here, in data verification, the MD5 value is a summary of a fixed length (usually a 32-bit hexadecimal number) obtained by performing a hash operation on the original data. This summary can be regarded as the "fingerprint" of the data, because any slight change to the original data will result in a different MD5 value.

[0131] After calculating the above A_c and B_c, it can be calculated whether A_c=B_c is established. If so, it means that the migrated target data in the target volume B is consistent with the source volume A. If not, it can be determined that the migrated target data in the target volume B is inconsistent with the source volume A. At this time, the verification result meets the alarm condition and generates an alarm message to indicate that the target data should be re-migrated to avoid data loss.

[0132] In an embodiment of the present invention, it is possible to verify whether the migration result meets the alarm condition during the data migration process, without having to perform data verification after the migration is completed. This solves the problem of difficulty in performing data verification during ongoing business processes, ensures the security and reliability of data during the entire migration process, and caters to the current concept of fast storage and efficient storage.

[0133] In some optional implementations, the above step S32 includes:

[0134] Step b1, obtaining the first shard sequence number of the data shard corresponding to the migration progress.

[0135] Step b2: when the difference between the first shard sequence number and the second shard sequence number corresponding to the target data exceeds a preset threshold, determine that the migration progress meets the verification condition, and verify the target data.

[0136] As can be seen from the above, in the present invention, data verification is performed on the data slices that have completed the migration during the data migration process. Therefore, in order to avoid conflicts between the verification progress and the data migration progress, the verification progress can be made to lag behind the migration progress. For example, the above preset threshold can be set to 10 to ensure that the verification progress lags behind the migration progress by 10 data slices.

[0137] When implementing it, Figure 3 The figure shows a schematic diagram of the data verification process, in which the sequence number of the data slice currently in the M state can be determined as the first slice sequence number. At the same time, the next data slice currently in the C state can be determined as the data slice corresponding to the target data, and the sequence number of the data slice is determined as the second slice sequence number. For example, if the slice sequence number of the data slice currently in the C state is i, and the sequence number of the data slice in the M state is n, then the first slice sequence number can be n, and the second slice sequence number can be i+1.

[0138] Next, it can be determined whether the migration progress meets the verification condition. Specifically, the difference a between the first shard sequence number and the second shard sequence number corresponding to the target data can be calculated, where a=n-(i+1). Here, if the preset threshold is 10. Then, if a>10, it is determined that the migration progress meets the verification condition, and the target data in data shard i is verified; if a<10, it is determined that the migration progress does not meet the verification condition, and the verification progress is suspended, and a is repeatedly verified until a>10 when the target data is verified.

[0139] In an embodiment of the present invention, data verification is performed on the data shards that have completed the migration during the data migration process. Therefore, in order to avoid conflicts between the verification progress and the data migration progress, the verification progress can lag behind the migration progress, thereby solving the problem of difficulty in performing data verification during continuous business processes, ensuring the security and reliability of data during the entire migration process, and catering to the current concepts of fast storage and efficient storage.

[0140] In summary, in an embodiment of the present invention, first, based on the data slices in the source volume, the source volume data can be migrated to the target volume, and the mapping information corresponding to the data slices can be obtained, wherein the mapping information is used to indicate the access status of the data slices. Next, after detecting a processing request for the target data in the data slices, the access status of the target data can be obtained based on the mapping information. Then, when the access status meets the execution condition of the processing request, the target data can be processed, thereby avoiding the migration and other processing of the target data when the target data is in an access state that does not meet the execution condition, such as writing data, so as to maintain the consistency of the data in the source volume and the target volume, and reduce data loss during the data migration process.

[0141] In this embodiment, a data access control device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0142] This embodiment provides a data access control device, such as Figure 4 As shown, including:

[0143] Migration module 401, used to migrate source volume data to a target volume based on data slices in the source volume;

[0144] A first acquisition module 402 is used to acquire mapping information corresponding to the data shard, wherein the mapping information is used to indicate the access status of the data shard;

[0145] A second acquisition module 403 is used to acquire the access status of the target data based on the mapping information after detecting a processing request for the target data in the data shard;

[0146] The processing module 404 is used to process the target data when the access status meets the execution condition of the processing request.

[0147] In some optional implementations, the migration module 401 is further configured to:

[0148] After detecting the data migration instruction, determining the source storage pool and the target storage pool indicated by the data migration instruction, and determining the source volume according to the source storage pool;

[0149] Read the attribute information of the source volume;

[0150] Based on the attribute information, create a target volume corresponding to the source volume in the target storage pool;

[0151] Obtain the historical load data and volume capacity corresponding to the source volume;

[0152] Calculate the load average corresponding to the source volume based on the historical load data volume;

[0153] Determine the data capacity of the data shard based on a preset functional relationship, a load average and a volume capacity, wherein in the preset functional relationship, the data capacity of the data shard is inversely proportional to the load average, and the data capacity of the data shard is directly proportional to the volume capacity;

[0154] Determine the number of shards of the data shard based on the ratio of the volume capacity to the data capacity of the data shard;

[0155] Generate mapping information based on the number of shards;

[0156] Based on data sharding, perform data migration operations on the source volume data.

[0157] In some optional implementations, the preset functional relationship includes: an inverse proportional function; the migration module 401 is further used for:

[0158] Get the preset load constant;

[0159] Determine the independent variable of the inverse proportional function based on the sum of the load constant and the load mean;

[0160] The data capacity of the data slice is determined based on the product of the volume capacity and the inverse proportional function.

[0161] In some optional implementations, the access state includes: a first state, a second state, and a third state, wherein the first state is used to indicate that the target data is being migrated to the target volume, the second state is used to indicate that data is being written to the data slice corresponding to the target data, and the third state is used to indicate that the data slice corresponding to the target data is in a stateless state; the processing module 404 is further used to:

[0162] determining whether the access state is a third state;

[0163] If not, repeatedly monitor the access status corresponding to the target data until the access status is a third state;

[0164] If so, it is determined that the access status satisfies the execution condition, and the target data is processed in response to the processing request, and the access status corresponding to the target data in the mapping information is modified when executing the processing request.

[0165] In some optional embodiments, the processing request includes: a first request, a second request, and a third request, wherein the first request is used to migrate the target data to the target volume, the second request is used to write data in the data slice corresponding to the target data, and the third request is used to read the target data.

[0166] In some optional implementations, the processing module 404 is further configured to:

[0167] When the processing request is the third request, obtaining a task queue and determining whether the task queue is empty, wherein the task queue is used to cache processing requests for target data;

[0168] If not, cache the third request to the end of the task queue;

[0169] If so, in response to the third request, the target data is read.

[0170] In some optional embodiments, the device is also used for:

[0171] After detecting a fourth request for the target data, obtaining a migration progress for the source volume data, wherein the fourth request is used to instruct to verify the target data;

[0172] When the migration progress meets the verification conditions, the target data is verified, and when the verification result meets the alarm conditions, an alarm message is generated.

[0173] In some optional embodiments, the device is also used for:

[0174] Get the first shard sequence number of the data shard corresponding to the migration progress;

[0175] When the difference between the first shard sequence number and the second shard sequence number corresponding to the target data exceeds a preset threshold, it is determined that the migration progress meets the verification condition, and the target data is verified.

[0176] The data access control device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0177] The embodiment of the present invention also provides a computer device having the above Figure 4 The data access control device shown.

[0178] See also Figure 5 , Figure 5 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 5As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 5 A processor 10 is taken as an example.

[0179] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0180] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0181] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0182] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0183] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0184] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0185] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0186] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the accompanying description.

Claims

1. A data access control method, characterized in that: The method comprises: Migrate source volume data to the target volume based on data shards in the source volume; Acquire mapping information corresponding to the data shard, wherein the mapping information is used to indicate an access status of the data shard; After detecting a processing request for target data in the data shard, obtaining an access status of the target data based on the mapping information; When the access status satisfies an execution condition of the processing request, the target data is processed.

2. The method according to claim 1, characterized in that The step of migrating the source volume data to the target volume based on the data slices in the source volume includes: After detecting the data migration instruction, determining the source storage pool and the target storage pool indicated by the data migration instruction, and determining the source volume according to the source storage pool; Reading attribute information of the source volume; Based on the attribute information, creating a target volume corresponding to the source volume in the target storage pool; Obtaining the historical load data volume and volume capacity corresponding to the source volume; Based on the historical load data volume, calculating the load mean corresponding to the source volume; Determine the data capacity of the data shard based on a preset functional relationship, the load mean and the volume capacity, wherein in the preset functional relationship, the data capacity of the data shard is inversely proportional to the load mean, and the data capacity of the data shard is directly proportional to the volume capacity; Determining the number of shards of the data shard based on a ratio of the volume capacity to the data capacity of the data shard; Generate mapping information according to the number of shards; Based on the data slices, a data migration operation is performed on the source volume data.

3. The method according to claim 2, characterized in that The preset functional relationship includes: an inverse proportional function; The determining the data capacity of the data slice based on the preset functional relationship, the load mean and the volume capacity includes: Get the preset load constant; Determining an independent variable of the inverse proportional function based on the sum of the load constant and the load mean; The data capacity of the data slice is determined based on the product of the volume capacity and the inverse proportional function.

4. The method according to claim 1, characterized in that: The access status includes: a first status, a second status, and a third status, wherein the first status is used to indicate that the target data is being migrated to the target volume, the second status is used to indicate that data is being written to the data slice corresponding to the target data, and the third status is used to indicate that the data slice corresponding to the target data is in a stateless state; When the access status satisfies the execution condition of the processing request, processing the target data includes: Determining whether the access state is a third state; If not, repeatedly monitoring the access state corresponding to the target data until the access state is a third state; If so, determine that the access status satisfies the execution condition, process the target data in response to the processing request, and modify the access status corresponding to the target data in the mapping information when executing the processing request.

5. The method according to claim 1, characterized in that The processing request includes: a first request, a second request and a third request, wherein the first request is used to migrate the target data to the target volume, the second request is used to write data in the data slice corresponding to the target data, and the third request is used to read the target data.

6. The method according to claim 5, characterized in that When the access status satisfies the execution condition of the processing request, processing the target data further includes: When the processing request is the third request, obtaining a task queue and determining whether the task queue is empty, wherein the task queue is used to cache the processing request for the target data; If not, caching the third request to the end of the task queue; If so, in response to the third request, the target data is read.

7. The method according to claim 1, characterized in that The method further comprises: After detecting a fourth request for the target data, obtaining a migration progress for the source volume data, wherein the fourth request is used to instruct to verify the target data; When the migration progress meets the verification condition, the target data is verified, and when the verification result meets the alarm condition, an alarm message is generated.

8. The method according to claim 7, characterized in that When the migration progress satisfies the verification condition, verifying the target data includes: Obtain the first shard sequence number of the data shard corresponding to the migration progress; When the difference between the first shard sequence number and the second shard sequence number corresponding to the target data exceeds a preset threshold, it is determined that the migration progress meets the verification condition, and the target data is verified.

9. A data access control device, characterized in that: The device comprises: A migration module, used to migrate source volume data to a target volume based on data slices in the source volume; A first acquisition module, used to acquire mapping information corresponding to a data slice, wherein the mapping information is used to indicate an access status of the data slice; A second acquisition module, configured to acquire an access status of the target data based on the mapping information after detecting a processing request for the target data in the data shard; A processing module is used to process the target data when the access status meets the execution condition of the processing request.

10. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the data access control method according to any one of claims 1 to 8 by executing the computer instructions.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the data access control method according to any one of claims 1 to 8.

Images