Data writing control method and device, storage medium and electronic device

Abstract

The present application discloses a data writing control method and device, a storage medium and an electronic device, and relates to the field of computer technology. The method comprises receiving a target writing request; detecting a data relationship between target data and written data in the memory according to data writing information of the memory, and obtaining a detection result; when the detection result indicates that the data relationship is that reference data in the written data has a similar relationship with the target data, matching the target data and the reference data with respect to data content to obtain a matching result; and writing the target data into the memory according to the matching result, thereby solving the technical problem of low control efficiency of data writing in related technologies and achieving the technical effect of improving the control efficiency of data writing.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data writing control method and device, a storage medium, and an electronic device. Background Art

[0002] With the development of digital technology, the application of memory has become increasingly popular. The data generated by users is stored in the memory. In order to improve the effective use of the storage space of the memory, technical personnel in this field often perform data deduplication detection on the data to be written into the memory, thereby avoiding duplicate data from being written into the memory. Current data deduplication detection often adopts the method of data consistency detection. After receiving the data to be written, fingerprint calculation is performed on the data to be written and the stored data respectively to obtain fingerprint information representing the data content. Based on the fingerprint information, data consistency detection is performed on the data to be written and the data stored in the memory. Although this method can avoid writing duplicate data into the memory, this process consumes a large amount of memory running memory, resulting in low control efficiency of data writing into the memory. Summary of the Invention

[0003] The present application provides a data writing control method and device, a storage medium and an electronic device to at least solve the technical problem of low data writing control efficiency in related technologies.

[0004] The present application provides a data writing control method, comprising: receiving a target write request, wherein the target write request is used to request writing target data in a memory; detecting a data relationship between the target data and the written data in the memory based on data write information of the memory, and obtaining a detection result, wherein the data write information is used to indicate a data write status of the written data on the memory; when the detection result indicates that the data relationship is that there is reference data in the written data that has a similar relationship with the target data, performing data content matching on the target data and the reference data to obtain a matching result; and writing the target data into the memory based on the matching result.

[0005] The present application also provides a data writing control device, comprising: a first receiving module, used to receive a target write request, wherein the target write request is used to request to write target data in a memory; a first detection module, used to detect the data relationship between the target data and the written data in the memory according to the data write information of the memory, and obtain a detection result, wherein the data write information is used to indicate the data write status of the written data on the memory; a matching module, used to match the target data and the reference data for data content, and obtain a matching result, when the detection result indicates that the data relationship is that there is reference data in the written data that has a similar relationship with the target data; and a writing module, used to write the target data into the memory according to the matching result.

[0006] The present application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any of the above-mentioned data writing control methods when executing the computer program.

[0007] The present application also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned data writing control methods are implemented.

[0008] The present application also provides a computer program product, including a computer program, which implements the steps of any of the above-mentioned data writing control methods when the computer program is executed by a processor.

[0009] Through the present application, when there is target data to be written, the data relationship between the target data and the data already written in the memory is detected by using the data write information of the memory, and then after the data relationship indicates that there is a similar relationship between the reference data and the target data in the written data, the target data and the reference data are subjected to data content detection, thereby avoiding executing the data content detection operation for all the data to be written and each written data stored in the memory, thereby reducing the computing pressure of the memory. Therefore, the technical problem of low control efficiency of data writing in the related technology can be solved, and the technical effect of improving the control efficiency of data writing can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0011] Figure 11 is a hardware structure block diagram of a data writing control method according to an embodiment of the present application;

[0012] Figure 2 is a flow chart of data writing control according to an embodiment of the present application;

[0013] Figure 3 is a schematic diagram of an optional data multiple mapping relationship according to an embodiment of the present application;

[0014] Figure 4 is an optional data writing flow chart according to an embodiment of the present application;

[0015] Figure 5 This is an optional data deduplication flow chart implemented according to the present application;

[0016] Figure 6 This is a structural block diagram of a data writing control device according to an embodiment of the present application. DETAILED DESCRIPTION

[0017] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0018] It should be noted that, in the description of this application, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. The terms "first," "second," etc., in this application are used to distinguish similar objects, and are not used to describe a particular order or sequence.

[0019] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0020] In conjunction with the specific application environment architecture or specific hardware architecture on which the execution of the data writing control method depends, the specific application environment architecture or specific hardware architecture is described here.

[0021] The method embodiments provided in the embodiments of the present application can be executed in a server device or a similar computing device. Taking running on a server device as an example, Figure 1 : is a hardware structure block diagram of the data writing control method of the embodiment of the present application. Figure 1As shown, the server device may include one or more ( Figure 1 Only one is shown) a processor 102 (the processor 102 may include but is not limited to a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. The server device may also include a transmission device 106 and an input / output device 108 for communication functions. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above server device. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0022] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the startup method of the operating system in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implementing the above-mentioned method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely located relative to the processor 102, and these remote memories can be connected to the server device via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0023] Transmission device 106 is used to receive or transmit data via a network. A specific example of the aforementioned network may include a wireless network provided by a communication provider of the server device. In one embodiment, transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0024] An embodiment of the present application provides a method for controlling data writing, and the method is described in detail in conjunction with the execution flow of the method for controlling data writing.

[0025] The following is an explanation of the professional terms that appear in this application:

[0026] SSD (Solid State Disk);

[0027] NVMe (Non-volatile Memory Express);

[0028] Dedup (Deduplication, data deduplication);

[0029] FTL (Flash Translation Layer);

[0030] GC (Garbage Collection).

[0031] In this embodiment, a data writing control method is provided. Figure 2 is a flow chart of data writing control according to an embodiment of the present application, such as Figure 2 As shown, the method includes the following steps:

[0032] Step S202: receiving a target write request, wherein the target write request is used to request writing target data into a memory;

[0033] Step S204: detecting a data relationship between the target data and the written data in the memory according to the data writing information of the memory, and obtaining a detection result, wherein the data writing information is used to indicate a data writing status of the written data in the memory;

[0034] Step S206, when the detection result indicates that the data relationship is that reference data in the written data has a similar relationship with the target data, performing data content matching on the target data and the reference data to obtain a matching result;

[0035] Step S208: writing the target data into the memory according to the matching result.

[0036] Through the above steps, when there is target data to be written, the data relationship between the target data and the data already written in the memory is detected by using the data write information of the memory, and then after the data relationship indicates that there is a similar relationship between the reference data and the target data in the written data, the target data and the reference data are subjected to data content detection, thereby avoiding executing the data content detection operation for all the data to be written and each written data stored in the memory, thereby reducing the computing pressure of the memory. Therefore, the technical problem of low control efficiency of data writing in the related technology can be solved, and the technical effect of improving the control efficiency of data writing can be achieved.

[0037] In the embodiment provided in step S204, the data write information is information used to indicate the data write status of the data written in the server. The data write information may include, but is not limited to, time information and data attribute information, wherein the time information is used to indicate the writing time of the written data on the memory, and the data attribute information is used to indicate the data characteristics of the written data. The data attribute information may include, but is not limited to, data type, data size, etc., and this solution does not limit this.

[0038] Optionally, in an embodiment of the present application, the data write information is information used to characterize the data correlation between the written data and the target data to be written in the memory, and then the data correlation reflects the similarity between the written data and the target data in terms of data content. Therefore, the method of detecting the data relationship between the target data and the written data based on the data write information can be to detect the data correlation between the target data and the written data in the memory based on the data write information. When the data correlation is greater than or equal to the target correlation, it is determined that the current written data is the reference data that has a similar relationship with the target data in terms of data content. Furthermore, the data write information may include, but is not limited to, multiple sub-information, and the multiple sub-information is used to characterize the data correlation between the written data and the target data in the memory from multiple dimensions. The sub-information may be, but is not limited to, the writing time of the data in the memory, the amount of data, the data type, etc., and this solution does not limit this. Therefore, the scheme for detecting the data correlation between the target data and the written data in the memory based on the data writing information may be, but is not limited to: calculating the first correlation between the first writing time of the target data on the memory and the second writing time of each of the written data on the memory, and calculating the second correlation between the first data amount of the target data and the second data amount of each of the written data, and calculating the third correlation between the first data type of the target data and the second data type of the written data, wherein the data writing information includes the second writing time, the second data amount and the second data type; obtaining weight parameters corresponding to the first correlation, the second correlation and the third correlation, wherein the weight parameters are used to indicate the degree of influence of the correlation of the corresponding data feature dimension on the data correlation relationship between the target data and the written data on the memory; using the weight parameters to perform weighted sum calculation on the first correlation, the second correlation and the third correlation to obtain the data correlation. In the above manner, by using the attribute information of the data, the data content similarity between the target data and the written data is detected, thereby avoiding the operation of performing data fingerprint information calculation on the target data to be written and all the written data in order to detect whether the data is duplicate data in the related art, thereby reducing the operating load of the memory.

[0039] In the embodiment provided in step S206, the method of matching the data content of the target data and the reference data can be a method of directly matching the data content of the target data and the data content of the reference data, or a method of matching the fingerprint information of the target data and the reference data, and then the fingerprint information of the target data can be calculated to obtain the first fingerprint information for characterizing the data content of the target data, and the fingerprint information of the reference data can be calculated to obtain the second fingerprint information for characterizing the data content of the reference data, and then the first fingerprint information and the second fingerprint information are matched. When the matches are consistent, the target data can be determined to be duplicate data. When the first fingerprint information and the second fingerprint information do not match each other, the target data can be determined to be non-duplicate data.

[0040] In the embodiment provided in step S208, whether the target data indicated by the matching result is the same as the data content of the reference data is determined to determine whether the target data is duplicate data, and then whether the data needs to be written to the memory is determined. That is, when the target data is duplicate data, the operation of writing the target data to the memory is skipped; when the target data is not duplicate data, the target data is written to the memory.

[0041] As an optional implementation manner, detecting the data relationship between the target data and the written data in the memory according to the data writing information of the memory to obtain the detection result includes:

[0042] Acquire data attribute information of the target data carried by the target write request;

[0043] detecting target data similarity between the target data and each of the written data according to the data attribute information and the data writing information;

[0044] The written data whose target data similarity is greater than or equal to a target threshold is determined as the reference data.

[0045] Optionally, in an embodiment of the present application, data attribute information is information used to characterize the write status of target data in the memory. The data attribute information can reflect the data correlation between the target data and the data stored in the memory, and then reflect the similarity between the data content of the target data and the data content of the stored data. In this embodiment, the data attribute information may include but is not limited to time information and data feature information, wherein the time information is used to indicate the write time of the written data on the memory, and the data feature information may include but is not limited to data type, data size, etc., which is not limited in this solution.

[0046] Optionally, in an embodiment of the present application, the method of detecting the target data similarity between the target data and the written data based on the data attribute information and the data writing information can be to directly calculate the similarity between the data attribute information and the data writing information, thereby calculating the information similarity between the data attribute information and the data writing information, and then reflecting the similarity of the data content between the target data and the written data through the information similarity, thereby obtaining the target data similarity. This solution does not limit this.

[0047] Based on the above, we adopted a target data similarity detection method based on data attribute information and write time. By calculating the temporal proximity between the target data and the written data, as well as their similarity in size and type, we can comprehensively assess the similarity between the two. This method not only improves the accuracy of data deduplication but also reduces unnecessary data matching operations, thereby improving the performance of the entire system.

[0048] As an optional implementation manner, detecting the target data similarity between the target data and each reference data according to the data attribute information and the data writing information includes:

[0049] Calculating a time difference between a first writing time of the target data on the memory and a second writing time of each of the written data on the memory, wherein the data attribute information includes the first writing time and the data writing information includes the second writing time;

[0050] The time difference is converted into the target data similarity between the target data and the written data.

[0051] Optionally, in an embodiment of the present application, there is a certain correlation between the data writing time and the data content. Generally speaking, when the data writing time is close, the data may be, to some extent, duplicate data of some already written data.

[0052] Through the above content, by using the time when the data is stored in the memory to detect data similarity between the target data and the written data, unnecessary data matching operations in the data deduplication process are avoided, thereby improving the data deduplication efficiency.

[0053] As an optional implementation manner, performing data content matching on the target data and the reference data to obtain a matching result includes:

[0054] Calculating first fingerprint information of the target data, and obtaining second fingerprint information of the reference data stored in the memory, wherein the first fingerprint information is used to indicate data content of the target data, and the second fingerprint information is used to indicate data content of the reference data;

[0055] matching the first fingerprint information with the second fingerprint information;

[0056] When the first fingerprint information and the second fingerprint information match each other, it is determined that the reference data and the target data have the same data content.

[0057] Optionally, in an embodiment of the present application, the fingerprint information of the calculated data can be but is not limited to being calculated by hash calculation, that is, hash calculation is performed on the target data to obtain the first fingerprint information, and hash calculation is performed on the reference data to obtain the second fingerprint information.

[0058] As an optional implementation manner, obtaining the second fingerprint information of the reference data stored in the memory includes:

[0059] Acquire a target logical address of the data write information corresponding to the reference data, wherein the target logical address is used to indicate a storage location of the data write information of the reference data on a logical address page of the memory;

[0060] searching for a target physical address corresponding to the target logical address from a first mapping relationship, wherein the first mapping relationship records an association relationship between a logical address of each logical location for storing data write information and a physical address for storing fingerprint information of the data;

[0061] The second fingerprint information stored in the sub-storage area indicated by the target physical address is extracted from the target storage area of ​​the memory, wherein the target storage area is used to store the fingerprint information of each written data on the memory.

[0062] Optionally, in an embodiment of the present application, data write information is stored by using a logical address configured for each data on the memory, and a corresponding target physical address is configured for each data on the memory, and there is a one-to-one correspondence between the target physical address and the target logical address, and the fingerprint information of the data is stored using the target physical address.

[0063] Optionally, in an embodiment of the present application, a first mapping relationship is configured between the target logical address and the target physical address, and conversion between the target logical address and the target physical address is achieved through the first mapping relationship.

[0064] The above content utilizes the mapping relationship between logical addresses and physical addresses, a key step in data deduplication. Logical addresses are the addresses seen by the operating system or application, while physical addresses represent the actual location of the data on the storage device. By querying this mapping relationship, the system can quickly locate the actual storage location of the reference data, extract its fingerprint information for comparison, and improve the efficiency of data content comparison.

[0065] As an optional implementation, the method further includes:

[0066] receiving a data deduplication request, wherein the data deduplication request is used to instruct to clear duplicate data stored in a reference storage area;

[0067] In response to the data deduplication request, reading the data write information stored in the storage space corresponding to the multiple reference logical addresses corresponding to the reference storage area, wherein the multiple reference logical addresses are configured in a one-to-one correspondence with the multiple sub-reference storage areas included in the reference storage area, and each sub-reference storage area is used to store one data;

[0068] performing similarity detection on the data stored in the plurality of sub-reference storage areas according to the data writing information to obtain a plurality of first data having a similar relationship;

[0069] searching for a candidate physical address corresponding to the reference logical address of each of the first data from a first mapping relationship, wherein the first mapping relationship records an association relationship between the logical address of each logical location for storing data write information and the physical address for storing fingerprint information of the data;

[0070] A data deduplication operation is performed on the plurality of first data according to the third fingerprint information of the first data stored in the candidate physical addresses corresponding to the plurality of first data, wherein the third fingerprint information is used to indicate data content of the first data.

[0071] Optionally, in an embodiment of the present application, the method of performing similarity detection on data stored in multiple sub-storage areas based on data writing information may be a method of performing information similarity matching on data writing information of data in multiple storage areas, thereby obtaining information similarity between data writing information, and reflecting the similarity between data contents through this information similarity, thereby obtaining multiple first data with similar relationships.

[0072] Optionally, in an embodiment of the present application, the data storage area may be, but is not limited to, a storage area indicated by the logical address range of the memory, that is, the data deduplication request carries the logical address range of the data to be deduplicated, and the logical address range includes multiple logical addresses, each of which corresponds to a sub-reference storage area.

[0073] As described above, by responding to data deduplication requests, the system proactively deduplicates data within the storage area, a crucial step in improving storage efficiency. The system reads the data written to the storage area and performs a similarity check on the data, identifying those with high similarity. It then uses mapping relationships to locate the actual physical location of these data and further verify their fingerprint information to confirm whether any duplicate data exists.

[0074] As an optional implementation manner, performing a deduplication operation on a plurality of first data according to the third fingerprint information of the first data stored in the candidate physical address includes:

[0075] Filtering out a plurality of fourth fingerprint information having the same fingerprint information from the plurality of third fingerprint information;

[0076] Filtering the second data corresponding to the fourth fingerprint information from the plurality of first data;

[0077] Filtering out third data from which duplicates are to be deleted from the plurality of second data;

[0078] Configuring a data cleaning identifier for the third data in the memory, wherein the data cleaning identifier is used to indicate that the current data is duplicate data to be cleaned;

[0079] The data carrying the data cleaning identifier is searched from the reference storage area to perform the data deletion operation.

[0080] Optionally, in an embodiment of the present application, in the memory, duplicate data detection is performed on the stored data by using fingerprint information, and a data cleaning identifier is configured for the detected duplicate data. Subsequently, data cleaning identifier detection can be performed on the data stored in the memory, and the data carrying the data cleaning identifier can be used as duplicate data to be cleaned.

[0081] Optionally, in an embodiment of the present application, the second data is duplicate data with the same data content, and thus the second data content needs to be deduplicated, one of the multiple second data is retained for storage, and garbage data recovery operations are performed on the other data. In this embodiment, the third data filtered out from the multiple second data can be, but is not limited to, obtained by filtering based on the logical address sequence number of the data, that is, the second data with the smallest logical address among the multiple second data is used as the retained data, and the other data except the data with the smallest logical address is determined as the third data to be deleted.

[0082] Optionally, in an embodiment of the present application, a physical address can be configured for each written data in the memory, and the physical address is used to store a data cleaning identifier of the corresponding written data, that is, a logical address (used to store data writing information of the corresponding written data), a first physical address (used to store the data content of the corresponding written data), a second physical address (used to store the data cleaning identifier of the corresponding written data) and a fingerprint physical address (used to store the fingerprint information of the corresponding written data, that is, the target logical address mentioned above) are configured for each written data in the memory. There is a mapping relationship between the logical address and the physical address, thereby constructing a multiple mapping relationship in the memory. Figure 3 is a schematic diagram of an optional data multi-mapping relationship according to an embodiment of the present application, such as Figure 3 As shown, a mapping table of logical addresses (used to store data write information of written data, 6 logical addresses are shown in the figure, and each logical address corresponds to one written data) established in the memory based on data deduplication, a first physical address PBA1 (used to store written data, 6 first physical addresses are shown in the figure, and one written data is stored in the storage area indicated by each first physical address), a second physical address PBA2 (used to store a data cleanup identifier of written data, 6 second physical addresses are shown in the figure, and one written data cleanup identifier is stored in the storage area indicated by each second physical address, and the first physical address and the second physical address are configured correspondingly, that is, A corresponds to A', B corresponds to B', and C corresponds to C' in the figure), a fingerprint physical address FPBA (used to store fingerprint information of written data, 6 fingerprint physical addresses are shown in the figure, and one written data is stored in the storage area indicated by each fingerprint physical address, and the first physical address and the fingerprint physical address are configured correspondingly, that is, A corresponds to A'', B corresponds to B'', and C corresponds to C'' in the figure) and data additional field metadata is established.

[0083] A mapping relationship between LBAs and physical addresses (PBA1) is established using the first data mapping table FTL1. PBA2 is a newly added physical address entry used to store a data scrubbing identifier for written data. This identifier can be the physical address of the stored written data. Under normal non-deduplication conditions, PBA1 and PBA2 are identical. When PBA2 differs, it serves as a duplicate data identifier. The FPBA fingerprints the physical page address. A hash value is generated for each LBA user data, representing the data's fingerprint information (used to determine if the hash values ​​are equal to determine if the data is duplicated). The data hash value is stored in the NAND flash memory to improve the SSD's data deduplication efficiency. Therefore, a mapping relationship between fingerprint information and storage physical addresses (i.e., the second data mapping table FTL2) is established using fingerprint data FTL2, allowing the generated fingerprint information to be retrieved. Furthermore, to effectively improve data deduplication, a related parameter (RP) (i.e., data write information) is added to each LBA. When data is sent to the disk, the upper-layer storage system uses the RP parameter to indicate that data with higher duplication rates will have higher related parameters. For example, if the RP value is the same, the data duplication rate will be higher.

[0084] (1) Establish a host data tracking deduplication process. a. When data is written to the disk, fingerprint information is not generated immediately. Instead, the mapping relationship between LBA and physical address PBA1 needs to be established through data FTL1, and the RP information of LBA needs to be recorded. This will not affect the latency performance issues caused by normal data writing. Figure 4 This is an optional data writing flow chart according to an embodiment of the present application, such as Figure 4 As shown, when the index number of the first physical address corresponding to the logical address is greater than 1, it means that one first physical address corresponds to multiple logical addresses, and it is necessary to re-establish the mapping relationship between the logical address and the first physical address, that is, cancel the previous<LBA,PBA1> The mapping relationship and re-establish the data FTL1 mapping<LBA,PBA1'> Valid mapping is completed, and the number of host logical address indexes corresponding to PBA1, data_dup_cnt, is decremented.

[0085] Figure 5This is an optional data deduplication flow chart implemented according to the present application. After the SSD has been running for a period of time, the host sends the tracking user data deduplication LBA command SetDataDedup_LBA_Status. The command information contains the data LBA range [lba_start, lba_end] to be deduplicated and the corresponding time correlation parameter RP1. In this case, the device first searches the data FTL1 mapping table according to the RP1 parameter, and reads the lba and corresponding user data with the same RP1 time parameter of lba∈[lba_start, lba_end] into the data FTL1 mapping table. Cache data buffer, and generate corresponding fingerprint information through the fingerprint generation information acceleration module FPEngine, and compare the fingerprint information from lba_start to lba_end in sequence. If the fingerprint information of lba1 and lba2 is consistent, the lba with the larger logical address is deduplicated. lba_base is defined as the smaller lba with the same user data, and lba_dedup is defined as the lba of another duplicate user data, where lba_base < lba_dedup (assuming there are currently only 2 duplicates in the range [lba_start, lba_end]). Process as follows:

[0086] b.1) Modify the PBA1 address corresponding to the deduplicated lba_dedup to the PBA1 corresponding to lba_base, and invalidate the PBA2 data duplication flag and FPBA data fingerprint physical information corresponding to lba_dedup (PBA2 / FPBA can be used for data garbage collection later). In other words, modify the data FTL1 mapping table and fingerprint data FTL2 mapping table entries for duplicate lba_dedup;

[0087] b.2) Based on the steps in b.1), complete the processing of lba in all SetDataDedup_LBA_Status commands and complete the deduplication of tracking user data. When the data deduplication is completed, the 1-to-1 mapping relationship of the original data FTL1 mapping is<lba,pba1> Some will become a mapping of n lba to one pba1, that is<lba1,pba1> 、<lba2,pba1> …<lba_n,pba1> , and the data physical address pba1 has a corresponding index count data_dup_cnt.

[0088] (2) GC garbage collection after data deduplication: After the disk data has been running for a long time, the host-based data deduplication process will have a many-to-one mapping in the data FTL1 mapping. GC garbage collection needs to read the valid data and write it to the new physical address PBA1'. Since this process involves multiple lba1 / 2 / 3...lba_n corresponding to the same physical address mapping pba1 mapping, it is necessary to<lba1,pba1> ,<lba2,pba1> ,…<lba_n,pba1> The mapping is moved back to<lba1,pba1’> ,<lba2,pba1’> ,…<lba_n,pba1’> To reclaim a block, you need to use the index count data_dup_cnt of the data physical address pba. After each lba_x (x=1 / 2… / n) data move is completed, data_dup_cnt is decremented. When it is decremented to 0, the move of all valid data in the nand block is completed.

[0089] The embodiment of the present application constructs an architecture and method for offline deduplication of host-tracked data by establishing multiple data mapping table entries, host-tracking data deduplication commands, and a block index counting method for garbage collection by the device after deduplication. By establishing a data FTL1 mapping of a logical address and a data fingerprint FTL2 mapping, and adding additional metadata information such as the time parameter correlation RP of the mapping table entry, when the host issues a tracking user data deduplication command, the device uses the time correlation parameter in the deduplication command to traverse the corresponding logical address range, calculate and compare the corresponding data fingerprint information, complete the entire data deduplication, and establish a many-to-one mapping relationship. A GC garbage collection method is established after data deduplication, and an index count of the data physical address is established during the data deduplication process. When triggering the block recovery of Nand, this index count value is maintained to complete the effective data movement action. This invention can effectively improve the collaborative processing of the storage host and the device, and improve the effectiveness of data deduplication on the disk based on the time correlation parameter of the deduplication data.

[0090] Through the above, accurate fingerprint information comparison ensures that no non-duplicate data will be deleted by mistake. The system will screen the data fingerprint information in all candidate physical addresses to find data with completely consistent fingerprint information.

[0091] As an optional implementation manner, configuring a data cleaning flag for the third data in the memory includes:

[0092] searching, from a second mapping relationship, for an initial physical address corresponding to the candidate physical address storing the fourth fingerprint information, wherein the second mapping relationship records an association between the physical address storing the fingerprint information and the physical address of the data cleansing identifier corresponding to the data storing the fingerprint information;

[0093] Using the initial physical address, searching for an identification storage space on the memory configured for the third data and used for storing a cleaning identification;

[0094] The data cleaning identifier is stored in the identifier storage space.

[0095] Optionally, in this embodiment of the present application, configuring the data cleanup flag is a key step in ensuring the correct execution of the data deduplication operation in this embodiment. The system searches the second mapping relationship for the initial physical address corresponding to the fourth fingerprint information. This step ensures that the system can accurately locate the data that needs to be marked.

[0096] Through the above content, by configuring the data cleaning identifier for the data, it is only necessary to search for the data cleaning identifier carried by the data in the future to easily locate the data to be cleaned for data garbage collection.

[0097] As an optional implementation manner, writing the target data into the memory according to the matching result includes one of the following:

[0098] When the matching result indicates that the reference data and the target data have the same data content, skipping executing a write operation on the target data in the memory;

[0099] When the matching result indicates that the reference data and the target data have different data contents, the target data is written to the memory.

[0100] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method.

[0101] The embodiment of the present application also provides a data writing control device, Figure 6 is a structural block diagram of a data writing control device according to an embodiment of the present application, such as Figure 6 As shown, the device includes:

[0102] A first receiving module is configured to receive a target write request, wherein the target write request is used to request writing target data into a memory;

[0103] a first detection module, configured to detect a data relationship between the target data and the data already written in the memory according to data writing information of the memory, and obtain a detection result, wherein the data writing information is used to indicate a data writing status of the already written data in the memory;

[0104] A matching module, configured to, when the detection result indicates that the data relationship is that reference data exists in the written data and has a similar relationship with the target data, perform data content matching on the target data and the reference data to obtain a matching result;

[0105] A writing module is used to write the target data into the memory according to the matching result.

[0106] Through the above device, when there is target data to be written, the data relationship between the target data and the data already written in the memory is detected by using the data write information of the memory, and then after the data relationship indicates that there is a similar relationship between the reference data and the target data in the written data, the target data and the reference data are subjected to data content detection, thereby avoiding executing the data content detection operation for all the data to be written and each written data stored in the memory, thereby reducing the computing pressure of the memory. Therefore, it can solve the technical problem of low control efficiency of data writing in the related technology and achieve the technical effect of improving the control efficiency of data writing.

[0107] Optionally, the first detection module includes:

[0108] an acquiring unit, configured to acquire data attribute information of the target data carried by the target write request;

[0109] a detection unit, configured to detect target data similarity between the target data and each of the written data according to the data attribute information and the data writing information;

[0110] The first determining unit is configured to determine the written data having a similarity with the target data greater than or equal to a target threshold as the reference data.

[0111] Optionally, the detection unit is used to:

[0112] Calculating a time difference between a first writing time of the target data on the memory and a second writing time of each of the written data on the memory, wherein the data attribute information includes the first writing time and the data writing information includes the second writing time;

[0113] The time difference is converted into the target data similarity between the target data and the written data.

[0114] Optionally, the matching module includes:

[0115] a first processing unit, configured to calculate first fingerprint information of the target data, and obtain second fingerprint information of the reference data stored in the memory, wherein the first fingerprint information is used to indicate data content of the target data, and the second fingerprint information is used to indicate data content of the reference data;

[0116] a matching unit, configured to match the first fingerprint information with the second fingerprint information;

[0117] The second determining unit is configured to determine that the reference data and the target data have the same data content when the first fingerprint information and the second fingerprint information match each other.

[0118] Optionally, the processing unit is configured to:

[0119] Acquire a target logical address of the data write information corresponding to the reference data, wherein the target logical address is used to indicate a storage location of the data write information of the reference data on a logical address page of the memory;

[0120] searching for a target physical address corresponding to the target logical address from a first mapping relationship, wherein the first mapping relationship records an association relationship between a logical address of each logical location for storing data write information and a physical address for storing fingerprint information of the data;

[0121] The second fingerprint information stored in the sub-storage area indicated by the target physical address is extracted from the target storage area of ​​the memory, wherein the target storage area is used to store the fingerprint information of each written data on the memory.

[0122] Optionally, the device further includes:

[0123] A second receiving module is configured to receive a data deduplication request, wherein the data deduplication request is used to instruct to clear duplicate data stored in the reference storage area;

[0124] a reading module, configured to respond to the data deduplication request and read the data write information stored in the storage space corresponding to the multiple reference logical addresses corresponding to the reference storage area, wherein the multiple reference logical addresses are configured in a one-to-one correspondence with the multiple sub-reference storage areas included in the reference storage area, and each sub-reference storage area is used to store one data;

[0125] a second detection module, configured to perform similarity detection on the data stored in the plurality of sub-reference storage areas according to the data writing information, to obtain a plurality of first data having a similar relationship;

[0126] a search module, configured to search for a candidate physical address corresponding to the reference logical address of each of the first data from a first mapping relationship, wherein the first mapping relationship records an association between the logical address of each logical location for storing data write information and the physical address for storing fingerprint information of the data;

[0127] An execution module is configured to perform a deduplication operation on the plurality of first data according to the third fingerprint information of the first data stored in the candidate physical addresses corresponding to the plurality of first data, wherein the third fingerprint information is used to indicate the data content of the first data.

[0128] Optionally, the execution module includes:

[0129] a first screening unit, configured to screen out a plurality of fourth fingerprint information having the same fingerprint information from the plurality of third fingerprint information;

[0130] a second screening unit, configured to screen out second data corresponding to the fourth fingerprint information from the plurality of first data;

[0131] a third screening unit, configured to screen out third data from which duplicate data is to be deleted from the plurality of second data;

[0132] a configuration unit, configured to configure a data cleaning flag for the third data in the memory, wherein the data cleaning flag is used to indicate that the current data is duplicate data to be cleaned;

[0133] A search unit is used to search the reference storage area for data carrying the data cleaning identifier to perform the data deletion operation.

[0134] Optionally, the configuration unit is used to:

[0135] searching, from a second mapping relationship, for an initial physical address corresponding to the candidate physical address storing the fourth fingerprint information, wherein the second mapping relationship records an association between the physical address storing the fingerprint information and the physical address of the data cleansing identifier corresponding to the data storing the fingerprint information;

[0136] Using the initial physical address, searching for an identification storage space on the memory configured for the third data and used for storing a cleaning identification;

[0137] The data cleaning identifier is stored in the identifier storage space.

[0138] Optionally, the writing module includes one of the following:

[0139] a second processing unit, configured to skip executing a write operation on the target data in the memory if the matching result indicates that the reference data and the target data have the same data content;

[0140] A writing unit is configured to write the target data into the memory when the matching result indicates that the reference data and the target data have different data contents.

[0141] For the description of the features in the embodiment corresponding to the data writing control device, reference can be made to the relevant description of the embodiment corresponding to the data writing control method, which will not be repeated here.

[0142] An embodiment of the present application further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any of the above-mentioned data writing control method embodiments.

[0143] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any one of the above-mentioned XX method embodiments when run.

[0144] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0145] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of any of the above-mentioned data writing control method embodiments are implemented.

[0146] An embodiment of the present application also provides another computer program product, including a non-volatile computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps of any of the above-mentioned data writing control method embodiments.

[0147] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0148] The above is a detailed introduction to a data writing control method and device, storage medium and electronic device provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core ideas of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A data writing control method, characterized in that: include: receiving a target write request, wherein the target write request is used to request writing target data in a memory; detecting a data relationship between the target data and the written data in the memory according to data writing information of the memory, and obtaining a detection result, wherein the data writing information is used to indicate a data writing status of the written data in the memory; When the detection result indicates that the data relationship is that reference data exists in the written data and has a similar relationship with the target data, performing data content matching on the target data and the reference data to obtain a matching result; Writing the target data into the memory according to the matching result; The detecting, based on the data writing information of the memory, the data relationship between the target data and the already written data in the memory to obtain the detection result includes: obtaining data attribute information of the target data carried by the target write request; detecting target data similarity between the target data and each of the already written data based on the data attribute information and the data writing information; and determining the already written data whose target data similarity is greater than or equal to a target threshold as the reference data; The detecting the target data similarity between the target data and each of the reference data based on the data attribute information and the data writing information includes: calculating the time difference between the first writing time of the target data on the memory and the second writing time of each of the written data on the memory, wherein the data attribute information includes the first writing time and the data writing information includes the second writing time; and converting the time difference into the target data similarity between the target data and the written data.

2. The method according to claim 1, characterized in that The performing data content matching on the target data and the reference data to obtain a matching result includes: Calculating first fingerprint information of the target data, and obtaining second fingerprint information of the reference data stored in the memory, wherein the first fingerprint information is used to indicate data content of the target data, and the second fingerprint information is used to indicate data content of the reference data; matching the first fingerprint information with the second fingerprint information; When the first fingerprint information and the second fingerprint information match each other, it is determined that the reference data and the target data have the same data content.

3. The method according to claim 2, characterized in that The acquiring the second fingerprint information of the reference data stored in the memory includes: Acquire a target logical address of the data write information corresponding to the reference data, wherein the target logical address is used to indicate a storage location of the data write information of the reference data on a logical address page of the memory; searching for a target physical address corresponding to the target logical address from a first mapping relationship, wherein the first mapping relationship records an association relationship between a logical address of each logical location for storing data write information and a physical address for storing fingerprint information of the data; The second fingerprint information stored in the sub-storage area indicated by the target physical address is extracted from the target storage area of ​​the memory, wherein the target storage area is used to store the fingerprint information of each written data on the memory.

4. The method according to claim 1, wherein The method further comprises: receiving a data deduplication request, wherein the data deduplication request is used to instruct to clear duplicate data stored in a reference storage area; In response to the data deduplication request, reading the data write information stored in the storage space corresponding to the multiple reference logical addresses corresponding to the reference storage area, wherein the multiple reference logical addresses are configured in a one-to-one correspondence with the multiple sub-reference storage areas included in the reference storage area, and each sub-reference storage area is used to store one data; performing similarity detection on the data stored in the plurality of sub-reference storage areas according to the data writing information to obtain a plurality of first data having a similar relationship; searching for a candidate physical address corresponding to the reference logical address of each of the first data from a first mapping relationship, wherein the first mapping relationship records an association relationship between the logical address of each logical location for storing data write information and the physical address for storing fingerprint information of the data; A data deduplication operation is performed on the plurality of first data according to the third fingerprint information of the first data stored in the candidate physical addresses corresponding to the plurality of first data, wherein the third fingerprint information is used to indicate data content of the first data.

5. The method according to claim 4, characterized in that The performing a deduplication operation on the plurality of first data according to the third fingerprint information of the first data stored in the candidate physical address includes: Filtering out a plurality of fourth fingerprint information having the same fingerprint information from the plurality of third fingerprint information; Filtering the second data corresponding to the fourth fingerprint information from the plurality of first data; Filtering out third data from which duplicates are to be deleted from the plurality of second data; Configuring a data cleaning identifier for the third data in the memory, wherein the data cleaning identifier is used to indicate that the current data is duplicate data to be cleaned; The data carrying the data cleaning identifier is searched from the reference storage area to perform the data deletion operation.

6. The method according to claim 5, characterized in that The configuring a data cleaning identifier for the third data in the memory includes: searching, from a second mapping relationship, for an initial physical address corresponding to the candidate physical address storing the fourth fingerprint information, wherein the second mapping relationship records an association between the physical address storing the fingerprint information and the physical address of the data cleansing identifier corresponding to the data storing the fingerprint information; Using the initial physical address, searching for an identification storage space on the memory configured for the third data and used for storing a cleaning identification; The data cleaning identifier is stored in the identifier storage space.

7. The method according to claim 1, characterized in that Writing the target data into the memory according to the matching result includes one of the following: When the matching result indicates that the reference data and the target data have the same data content, skipping executing a write operation on the target data in the memory; When the matching result indicates that the reference data and the target data have different data contents, the target data is written to the memory.

8. A data writing control device, characterized in that: include: A first receiving module is configured to receive a target write request, wherein the target write request is used to request writing target data into a memory; a first detection module, configured to detect a data relationship between the target data and the data already written in the memory according to data writing information of the memory, and obtain a detection result, wherein the data writing information is used to indicate a data writing status of the already written data in the memory; A matching module, configured to, when the detection result indicates that the data relationship is that reference data exists in the written data and has a similar relationship with the target data, perform data content matching on the target data and the reference data to obtain a matching result; A writing module, configured to write the target data into the memory according to the matching result; The first detection module includes: an acquisition unit for acquiring data attribute information of the target data carried by the target write request; a detection unit for detecting target data similarity between the target data and each of the written data based on the data attribute information and the data write information; a first determination unit for determining the written data whose target data similarity is greater than or equal to a target threshold as the reference data; The detection unit is used to: calculate the time difference between the first writing time of the target data on the memory and the second writing time of each of the written data on the memory, wherein the data attribute information includes the first writing time and the data writing information includes the second writing time; and convert the time difference into the target data similarity between the target data and the written data.

9. An electronic device, characterized in that: include: memory for storing computer programs; A processor, configured to implement the steps of the data writing control method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the data writing control method according to any one of claims 1 to 7 are implemented.

Images