Data processing method and device, electronic equipment and storage medium
Through asynchronous operation of the file system and storage media, the problem of data processing time in storage media is solved, and rapid data processing and efficient device operation are achieved.
Patent Information
- Application Number
- CN202311850537.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-06-05
- Filing Date
- 2023-12-28
- Publication Date
- 2024-12-06
AI Technical Summary
The prior art In the storage medium of electronic devices, the data read and write interaction process requires a large amount of memory and data read and write operations, resulting in long data processing and delayed device operation.
Through asynchronous operation of the file system and storage media, the memory overhead of data processing and data input and output overhead are reduced, and data migration and discarding are directly carried out in the storage media to achieve rapid data processing.
Improves equipment work efficiency, reduces data processing time, and avoids equipment operation lag.
Smart Images

Figure CN119088286A_ABST
Abstract
Description
[0001] This disclosure claims the priority of the Chinese patent application filed with the China Patent Office on June 5, 2023, with application number 202310661364.3 and invention name “Data processing method and device, electronic device, storage medium”, all contents of which are incorporated by reference in this disclosure. Technical Field
[0002] The embodiments of the present application relate to the field of electronic technology, and relate to but are not limited to a data processing method and device, an electronic device, and a storage medium. Background Art
[0003] When data distribution is discontinuous or available space is fragmented inside the storage medium of the electronic device, the data needs to be sorted or discarded, etc. The prior art mainly processes the data inside the storage medium through the data reading and writing interaction process between the file system and the storage medium.
[0004] Since the data reading and writing interaction process requires the use of a large amount of file system memory and a large number of data reading and writing operations, the data processing process takes a long time and the electronic device runs slowly. Summary of the invention
[0005] In view of this, the data processing method and device, electronic device, and storage medium provided in the embodiments of the present application can complete the processing of internal data of the storage medium through asynchronous operations of the file system and the storage medium, reduce the memory overhead and data input and output overhead of data processing, realize fast data processing, and improve the working efficiency of the equipment.
[0006] In a first aspect, the data processing method provided in the embodiment of the present application is applied to a file system, and the method includes:
[0007] Acquire a source address and processing information of the data to be processed, wherein the source address is used to indicate a storage address of the data to be processed in a storage medium, and the processing information is used to instruct the storage medium to perform data migration and / or data discard on the data to be processed;
[0008] An information packet is generated according to the source address and the processing information and sent to the storage medium, so that the storage medium processes the data to be processed according to the information packet.
[0009] In a second aspect, a data processing method according to an embodiment of the present application is applied to a storage medium, and the method includes:
[0010] Acquire an information packet, the information packet including a source address of the data to be processed and processing information, the source address is used to indicate a storage address of the data to be processed in a storage medium, and the processing information is used to instruct the storage medium to perform data migration and / or data discard on the data to be processed;
[0011] The data to be processed is processed according to the information packet.
[0012] In a third aspect, the data processing device provided in the embodiment of the present application is applied to an electronic device, including:
[0013] A first acquisition module, used to acquire a source address and processing information of the data to be processed, wherein the source address is used to indicate a storage address of the data to be processed in a storage medium, and the processing information is used to instruct the storage medium to perform data migration and / or data discard on the data to be processed;
[0014] The information sending module is used to generate an information packet according to the source address and the processing information and send the information packet to the storage medium, so that the storage medium processes the data to be processed according to the information packet.
[0015] In a fourth aspect, the data processing device provided in the embodiments of the present application is applied to an electronic device, including:
[0016] A second acquisition module is used to acquire an information packet, wherein the information packet includes a source address of the data to be processed and processing information, wherein the source address is used to indicate a storage address of the data to be processed in a storage medium, and the processing information is used to instruct the storage medium to perform data migration and / or data discard on the data to be processed;
[0017] A data processing module is used to process the data to be processed according to the information packet.
[0018] In a fifth aspect, an electronic device provided in an embodiment of the present application includes a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, the steps of the data processing method provided in the first aspect or the second aspect of the embodiment of the present application are implemented.
[0019] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the data processing method provided in the first aspect or the second aspect of the embodiment of the present application are implemented.
[0020] The data processing method, device, electronic device and computer-readable storage medium provided in the embodiments of the present application can complete the processing of internal data of the storage medium through asynchronous operations of the file system and the storage medium, reduce the memory overhead and data input and output overhead of data processing, realize fast data processing, and improve the working efficiency of the equipment, thereby solving the technical problems raised in the background technology. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.
[0022] Figure 1 A flowchart of an existing data processing method provided in an embodiment of the present application;
[0023] Figure 2 A schematic diagram of an implementation flow of a data processing method provided in an embodiment of the present application;
[0024] Figure 3 A schematic diagram of a process for obtaining a target address provided in an embodiment of the present application;
[0025] Figure 4 A schematic diagram of a target address application provided in an embodiment of the present application;
[0026] Figure 5 A schematic diagram of an implementation flow of obtaining processing information provided in an embodiment of the present application;
[0027] Figure 6 A schematic diagram of an information packet structure provided in an embodiment of the present application;
[0028] Figure 7 A flowchart of another data processing method provided in an embodiment of the present application;
[0029] Figure 8 A schematic diagram of another information packet structure provided in an embodiment of the present application;
[0030] Fig. 9 A schematic diagram of a descriptor definition table and a device descriptor provided in an embodiment of the present application;
[0031] Fig.10 A schematic diagram of a parameter definition rule table for a data processing function provided in an embodiment of the present application;
[0032] Fig.11 A schematic diagram of an implementation flow of another data processing method provided in an embodiment of the present application;
[0033] Fig.12A schematic diagram of the format of a command description block provided in an embodiment of the present application;
[0034] Fig.13 A schematic diagram of data content of a command description block provided in an embodiment of the present application;
[0035] Fig.14 A flowchart of a method for migrating data to be processed provided in an embodiment of the present application;
[0036] Fig.15 A schematic diagram of a data header format of a command description block provided in an embodiment of the present application;
[0037] Fig.16 A schematic diagram of a format definition of a copy command parameter table of a command description block provided in an embodiment of the present application;
[0038] Fig.17 A schematic diagram of a format definition of a device block descriptor provided in an embodiment of the present application;
[0039] Fig.18 A schematic diagram of a data header format of another command description block provided in an embodiment of the present application;
[0040] Fig.19 A schematic diagram of a format definition of a copy command parameter table of another command description block provided in an embodiment of the present application;
[0041] Fig. 20 A schematic diagram of a format definition of a descriptor of another device block provided in an embodiment of the present application;
[0042] Fig.21 A schematic diagram of a flow chart of data processing by a storage medium provided in an embodiment of the present application;
[0043] Fig. 22 A schematic diagram of a data processing flow provided in an embodiment of the present application;
[0044] Fig.23 A schematic diagram of a method for processing data in a to-be-reclaimed area provided in an embodiment of the present application;
[0045] Fig.24 A schematic diagram of the overall flow of a data processing method provided in an embodiment of the present application;
[0046] Fig.25 A schematic diagram of a data processing method based on a target file provided in an embodiment of the present application;
[0047] Fig.26 A schematic diagram of a scenario of a data processing method based on type information provided in an embodiment of the present application;
[0048] Fig. 27 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application;
[0049] Fig.28 A schematic diagram of the structure of another data processing device provided in an embodiment of the present application;
[0050] Fig.29 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0051] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the specific technical solution of the present application will be further described in detail below in conjunction with the drawings in the embodiments of the present application. The following embodiments are used to illustrate the present application, but are not used to limit the scope of the present application.
[0052] In the current storage field, in order to ensure that data can be quickly saved in non-volatile storage, more and more storage media choose to use off-site updates to save data. Compared with the local update writing method, off-site updates can bring users a faster response experience, but it also causes discontinuity in the data distribution within the storage medium and fragmented distribution of available space. With the increasing discreteness of data within the storage medium, in order to ensure the continuous high-speed response of read and write performance and the unified recovery management of available space, it is imperative to manage and organize valid data and clean up invalid data.
[0053] Figure 1 The following is a flow chart of an existing data processing method provided in an embodiment of the present application. Figure 1 As shown, the user or the system sends a data arrangement command for the target data, such as a data migration command, to the file system in the kernel state through the user state. The file system sends a read data command to the flash memory according to the data arrangement command. The flash memory reads the target data to the page cache memory, that is, stores it in the memory. The page cache memory then returns a read success command to the file system. After receiving the read success command, the file system sends a write data command to the page cache memory to write the target data to a new storage location. The page cache memory writes the target data to the new storage location in the flash memory according to the write data command. After the flash memory is written, a write success command is fed back to the file system.
[0054] As can be seen from the above process, before migrating data, the file system must first read all the target data into the memory for storage, which will inevitably increase the additional memory overhead. In the process of reading the target data and subsequently rewriting it into the flash memory, it will also inevitably introduce redundant data input and output IO overhead. In addition, a large number of interactive processing processes also cause data migration to take a long time. In general, the data reading and writing interactive process in the data processing process requires the use of a large amount of memory of the file system and a large number of data reading and writing operations, which will inevitably cause the processing process to take a long time and the electronic equipment to run slowly.
[0055] In view of this, an embodiment of the present application provides a data processing method, which can be applied to the file system of an electronic device, and the electronic device can be various types of devices with information processing capabilities during the implementation process. For example, the electronic device can include a personal computer, a laptop, a PDA or a server, etc.; the electronic device can also be a mobile terminal, for example, the mobile terminal can include a mobile phone, a car computer, a tablet computer or a projector, etc. The file system can be a log-structured file system LFS (Log-Structured File System), a new technology file system NTFS, a file allocation table file system FAT, a high-performance file system HPFS, etc. The method includes obtaining the source address and processing information of the data to be processed, the source address is used to indicate the storage address of the data to be processed in the storage medium, and the processing information is used to indicate that the storage medium performs data migration and / or data discard on the data to be processed; an information packet is generated according to the source address and the processing information and sent to the storage medium, so that the storage medium processes the data to be processed according to the information packet. In the method, the file system can obtain all the information required for the storage medium to process the data, generate an information packet and send it to the storage medium, so that the storage medium can directly process the data to be processed.
[0056] The embodiment of the present application also provides a data processing method, which is applied to the storage medium of an electronic device, and the storage medium can be a hard disk, a flash memory, a USB flash drive, etc. The method includes obtaining an information packet, the information packet including the source address of the data to be processed and processing information, the source address is used to indicate the storage address of the data to be processed in the storage medium, and the processing information is used to instruct the storage medium to perform data migration and / or data discard on the data to be processed; and processing the data to be processed according to the information packet. In the method, the storage medium can obtain the information packet sent by the file system, and process the data to be processed according to the information packet.
[0057] The above two data storage methods complete the processing of the internal data of the storage medium through the asynchronous operation of the file system and the storage medium, reduce the memory overhead and data input and output overhead of data processing, realize fast data processing, and improve the working efficiency of the equipment. The following is a detailed description of the two data processing methods.
[0058] Figure 2 The present invention provides a data processing method for implementing a flowchart, which is applied to the file system of an electronic device. Figure 2 As shown, the method may include the following steps 101 to 102:
[0059] Step 101: Acquire a source address and processing information of the data to be processed, wherein the source address is used to indicate a storage address of the data to be processed in a storage medium, and the processing information is used to instruct the storage medium to perform data migration and / or data discard on the data to be processed.
[0060] Among them, the method for obtaining the source address may be to search for the source address of the data to be processed through the file system or through other methods. The embodiment of the present application does not limit the method for obtaining the source address of the data to be processed. Processing information is information that instructs the storage medium to perform data processing. Exemplarily, obtaining the processing information may be to obtain the target address of the data to be processed on the storage medium, so that the storage medium can perform data migration according to the source address and the target address. Exemplarily, obtaining the processing information may also be to obtain the data processing method corresponding to the data to be processed, so that the storage medium can perform data processing according to the source address and the data processing method, such as data discarding. The embodiment of the present application does not limit the method for obtaining the source address of the data to be processed and the processing information.
[0061] Step 102: Generate an information packet according to the source address and the processing information and send it to the storage medium, so that the storage medium processes the data to be processed according to the information packet.
[0062] It should be noted that the method of generating an information packet according to the source address and the processing information and sending it to the storage medium may be to generate a data packet according to the type of file system, the configuration of the device, or according to a preset method and send it to the storage medium. The embodiment of the present application does not limit the method of generating an information packet according to the source address and the processing information and sending it to the storage medium.
[0063] It is understandable that the information packet generated and sent by the file system includes all the information required by the storage medium for data processing. Therefore, after the storage medium receives the information packet, it can directly process the data according to the information packet. This method decouples the file system's dependence on the storage medium during data processing, and the file system is completely responsible for the operation of obtaining the source address of the data to be processed and processing information and generating the information packet, reducing the memory overhead and data input and output overhead of data processing, achieving fast data processing, and improving the working efficiency of the device.
[0064] The data processing method provided in the embodiments of the present application can be applied to different types of data processing scenarios. For example, a defragmentation scenario for single file data, or a defragmentation scenario for file system data, or a data migration scenario for multiple data of different data types or data attributes, etc. Exemplarily, it can be applied to a data migration scenario including hot data and cold data, and the hot data and the cold data are stored in different storage areas, where hot data refers to frequently used data and cold data refers to infrequently used data. It can also be applied to a data migration scenario including multiple types of data such as movie data, music data, short video data and scattered data, and different types of data are stored in different storage areas, etc. The embodiments of the present application do not limit the application scenarios of the data processing method.
[0065] In some embodiments, in step 101 , the file system may obtain the source address corresponding to the data to be processed by reading it from a preset storage space.
[0066] In an embodiment of the present application, obtaining the source address corresponding to the data to be processed may include: obtaining the source address from a preset storage space of the file system, wherein the source address is stored in the preset storage space after the data to be processed is found.
[0067] It should be noted that the source address of the data to be processed in the storage medium is also called the storage address corresponding to the data to be processed, which can be stored in the pre-applied memory space for the convenience of subsequent generation of information packets.
[0068] For example, if the file system adopts a log structure file system, for the data that needs to be sorted, that is, the data to be processed, it is first necessary to find the address of the data to be processed on the file system side, which can be collectively referred to as the source address. At this time, these source addresses should be partially continuous and partially non-continuous. After finding these source addresses, they can be saved in a temporary storage space. These source addresses found can be collectively referred to as a source address space set. The use of the above method to obtain the source address corresponding to the data to be processed takes advantage of the structural advantages of the file system and improves the working efficiency of the device.
[0069] It is understandable that for log-structured file systems, due to the different types of systems, the address management methods are bound to be different. However, in the embodiment of the present application, it is only necessary to find the storage address that stores the data to be processed and determine that it is the required source address. The previous level index of the source address can be obtained according to demand.
[0070] In some embodiments, in step 101, the processing information can be divided into two modes according to the different information included in the processing information. The first mode may include: a target address, and the target address is used to instruct the storage medium to perform data migration and / or data discard on the data to be processed; the second mode may include: a target address and tag information, and the tag information is used to instruct the storage medium to perform data migration and / or data discard on the data to be processed, and the target address is used to indicate the destination storage address when the storage medium performs data migration on the data to be processed. The following is a detailed description based on the two modes.
[0071] In a first approach, the processing information includes a target address, and the target address is used to instruct the storage medium to perform data migration and / or data discard on the data to be processed.
[0072] It should be noted that different contents of the target address can be set to instruct the storage medium to migrate or discard the data to be processed. The target address can be obtained by the file system assigning a new storage address to the data to be processed or by other means. The embodiment of the present application does not limit the method of obtaining the target address.
[0073] Figure 3 The present invention provides a schematic diagram of a process for obtaining a target address. Figure 3 As shown, the processing information includes a target address, and the acquisition of processing information of the data to be processed may include:
[0074] Step 201: Determine a data processing method corresponding to the data to be processed.
[0075] It should be noted that the data method corresponding to the data to be processed can be obtained by receiving a data processing command issued by the system, or by other methods. The embodiment of the present application does not limit the method for determining the data processing method corresponding to the data to be processed.
[0076] Step 202: If the data processing method is data discard, determine that the target address is the source address.
[0077] It should be noted that the file system may pre-agree with the storage medium that when the source address is the same as the target address, it means that the data to be processed is discarded.
[0078] Step 203: If the data processing method is data migration, determine that the target address corresponds to the destination storage address of the storage medium for migrating the data to be processed.
[0079] It is understandable that the file system can pre-agree with the storage medium that when the source address and the target address are different, it means that the data to be processed is migrated. When the data processing method is data migration, it is necessary to determine that the target address corresponds to the destination storage address of the storage medium to migrate the data to be processed.
[0080] Furthermore, determining in step 203 that the target address corresponds to the destination storage address of the storage medium for migrating the data to be processed may include:
[0081] Step 2031: Determine the target space in the storage medium according to the size of the source storage space of the data to be processed corresponding to the source address in the storage medium, wherein the size of the target space is greater than or equal to the size of the storage space corresponding to the source address. The size of the target space needs to be greater than or equal to the size of the storage space corresponding to the source address to ensure that the storage space is sufficient to store the data to be processed.
[0082] Step 2032: Determine the target address according to the address of the target space, the address of the target space is the destination storage address, and the target address has a one-to-one mapping relationship with the source address.
[0083] Exemplarily, after determining the source address, a target space can be applied for in the file system, and the target space points to the storage space in the storage medium. The target space can be continuous and of the same size as the source storage space of the to-be-processed data corresponding to the source address in the storage medium, or a continuous small-scale storage space whose sum is equal to the source storage space of the to-be-processed data corresponding to the source address in the storage medium, which is collectively referred to as the target address space set below. The target address is then determined according to the address of the target space, and the target address is the corresponding address in the file system.
[0084] Determining the target address in the above manner can ensure that the target address sufficiently corresponds to the source address, the address of the target space is the destination storage address, and the target address has a one-to-one mapping relationship with the source address, that is, when data is migrated, it is migrated from a source address to a corresponding target address. Among them, when the address in the source address is discontinuous, and the address in the determined target address is continuous, the data to be processed can be migrated from a discontinuous storage space to a continuous storage space.
[0085] Furthermore, generating an information packet according to the source address and the processing information and sending the information packet to the storage medium may include: generating an information packet according to the source address and the target address and sending the information packet.
[0086] It is understandable that after the file system obtains the target address, the target address and the source address are sent together, which can be collectively referred to as an information packet, so that the storage medium can process the data to be processed according to the information packet, thereby improving data processing efficiency.
[0087] In some embodiments, determining the target space in the storage medium according to the size of the source storage space in the storage medium of the data to be processed corresponding to the source address in step 201 may include: determining the storage space in the storage medium corresponding to the target log header in the file system as the target storage space, the storage space in the storage medium corresponding to the target log header being a preset storage space for storing the data to be processed; and determining a storage space from the target storage space that is greater than or equal to the size of the storage space corresponding to the source address as the target space.
[0088] Exemplarily, based on the structural characteristics of the file system as a log-structured file system, a mixed log header method can be used when selecting the target address, that is, according to actual needs, the storage space corresponding to the existing log header is selected for writing. However, the use of a mixed method is bound to interfere with the migration performance and efficiency. Therefore, the embodiment of the present application adopts a custom log header method, that is, a custom target log header is used to store the data to be processed, that is, the storage space in the storage medium corresponding to the target log header is first selected as the target storage space, and then the target address is determined from the target storage space. This method can uniformly manage the index of the target address and improve the reading and writing efficiency.
[0089] The file system allocates new space each time it writes data, whether it is appending or modifying. Therefore, the embodiment of the present application utilizes this feature to pre-apply for one or several consecutive spaces of the same size as the target address when organizing data.
[0090] In some embodiments, the file system is affected by the space allocation threshold when allocating storage space, that is, the file system can only allocate a maximum of storage space corresponding to the space allocation threshold each time.
[0091] In an embodiment of the present application, the target space may include at least two target subspaces, the size of each target subspace is equal to or less than a space allocation threshold, the space allocation threshold is the maximum storage space of the storage medium allocated by the file system at one time, and determining the address corresponding to the target space as the target address may include: determining the addresses corresponding to the at least two target subspaces as the target address.
[0092] It should be noted that when the storage space corresponding to the storage address is large, the target space that the file system needs to allocate is also large. The file system needs to allocate the target subspace multiple times according to the space allocation threshold to obtain at least two target subspaces. The address corresponding to the target subspace is the target address. The above method can obtain all target addresses corresponding to the storage address.
[0093] Figure 4 A schematic diagram of a target address application provided in an embodiment of the present application, such as Figure 4 As shown, the source address is from z to z+n, with a total of n+1 addresses. The target space allocated by the file system can be multiple target subspaces, where the address of the first target subspace is from c to c+m, the address of the second target subspace is from p to p+m, ..., the address of the tth target subspace is from x to x+m, and the address calculation satisfies the formula (m+1)*(t-1)<(n+1)≤(m+1)*t, where m+1 represents the number of addresses included in each target subspace, n+1 represents the number of source addresses, and t represents the number of target subspaces. Since the size of space allocated by the file system each time is limited by the space allocation threshold, the total number of addresses corresponding to the target subspace is likely to be greater than the number of addresses corresponding to the storage address, that is, the addresses of the last target subspace are not all used. Figure 4 As shown, the target space allocated by the file system can also be a continuous space, that is, the target address is n+1 addresses from a to a+n. It should be noted that the target address applied for is ideally as shown on the right side of the figure, that is, all addresses are continuous. However, the ideal situation may not be achieved in practice, so the target address currently applied for may only be guaranteed as shown in the middle of the figure, that is, the m addresses in each target address segment are continuous, but each segment of address may not be continuous.
[0094] Furthermore, when the target space includes multiple target subspaces, the generated information packet also needs to include multiple sub-information packets, and each sub-information packet includes the same number of source addresses and target addresses.
[0095] In an embodiment of the present application, generating an information packet according to the source address and the processing information and sending it to the storage medium may include: generating at least two sub-information packets according to the source address and the target address, each of the sub-information packets including a target number of the target addresses and a target number of the source addresses, the target number being less than or equal to the number of addresses corresponding to one of the target subspaces; and sending each of the at least two sub-information packets to the storage medium.
[0096] For example, if the storage address includes 1050 addresses and the file system can only apply for 100 target addresses each time, the file system needs to apply for the target address 11 times. After successfully applying for the first 100 addresses, the first 100 addresses of the source address and the first 100 addresses of the target address generate a sub-information packet and send it. After successfully applying for the second 100 addresses, the next 100 addresses of the source address and the second 100 addresses of the target address generate a sub-information packet and send it. And so on. When the last 100 addresses of the target address are applied for, the last 50 addresses of the storage address and 50 addresses of the last 100 addresses generate a sub-information packet and send it, and the extra addresses are not used. The above method can ensure that each source address has a corresponding target address to complete data migration.
[0097] In some embodiments, the data to be processed includes valid data and invalid data. When obtaining the target address of the data to be processed, it is only necessary to obtain the target address corresponding to the valid data.
[0098] In an embodiment of the present application, the data to be processed includes valid data and invalid data, and determining the target space in the storage medium according to the size of the source storage space of the data to be processed in the storage medium corresponding to the source address may include: determining the target space according to the size of the storage space of the valid data in the storage medium corresponding to the source address of the valid data in the source address.
[0099] For example, if the data to be processed includes 5 valid data and 5 invalid data, when determining the target address of the data to be processed, only the target addresses of the 5 valid data need to be determined, and the invalid data does not need to be migrated. Therefore, the determined target space is determined according to the size of the storage space corresponding to the source addresses of the 5 valid data. The method can realize data sorting of the storage space including both valid data and invalid data.
[0100] It is understandable that the data processing method provided in the embodiment of the present application is not limited to the processing of valid data, but is also applicable to invalid data. Because valid data is bound to contain a target address space set, while invalid data is the opposite. Therefore, the storage medium can identify the type of the source address space set by determining whether it contains a target address space set. For example, the target address corresponding to the source address of invalid data can be empty or 0.
[0101] Method 2: The processing information includes a target address and tag information, wherein the tag information is used to instruct the storage medium to migrate and / or discard the data to be processed, and the target address is used to indicate the destination storage address when the storage medium migrates the data to be processed.
[0102] It should be noted that different contents of the tag information can be set to instruct the storage medium to migrate or discard the data to be processed, and the target address is used to indicate the destination storage address of the data to be processed. The target address can be obtained by the file system assigning a new storage address to the data to be processed or by other methods, and the tag information can be obtained according to the correspondence between the preset processing method and the tag information. The embodiment of the present application does not limit the method of obtaining the target address and the tag information.
[0103] Figure 5 A schematic diagram of a process flow for obtaining processing information provided in an embodiment of the present application. Figure 5 As shown, the processing information includes the target address and tag information. The processing information of the data to be processed obtained in step 101 may include:
[0104] Step 301: Determine a data processing method corresponding to the data to be processed.
[0105] It should be noted that the data method corresponding to the data to be processed can be obtained by receiving a data processing command issued by the system, or by other methods. The embodiment of the present application does not limit the method for determining the data processing method corresponding to the data to be processed.
[0106] Step 302: Determine the tag information corresponding to the data to be processed according to the data processing method, and the data processing method and the tag information have a corresponding relationship.
[0107] It should be noted that the file system pre-stores a correspondence between processing methods and tag information. After determining the data processing method corresponding to the data to be processed, the tag information corresponding to the data to be processed can be obtained based on the correspondence between the data processing method and the tag information.
[0108] Step 303: When the data processing mode is data migration, determine that the target address corresponds to the destination storage address of the storage medium for migrating the data to be processed.
[0109] It should be noted that the processing information includes the target address and tag information. The storage medium can determine the data processing method according to the tag information. When the data processing method is determined to be data migration, the destination storage address for migrating the data to be processed is determined according to the target address.
[0110] Figure 6 A schematic diagram of an information packet structure provided in an embodiment of the present application. Figure 6 As shown, at the file system and storage medium end, a data packet structure for interaction can be customized. When both the source address and the target address have more than one address, the data packet can include a source address space set (multiple source addresses) and a target address space set (multiple target addresses) and a function flag bit (marking information corresponding to the source address), wherein the function flag bit is used to mark the data processing method of the data corresponding to the storage address.
[0111] In some embodiments, determining in step 303 that the target address corresponds to the destination storage address of the storage medium for migrating the data to be processed may include:
[0112] Step 3031: determining a target space in the storage medium according to the size of the source storage space of the data to be processed corresponding to the source address in the storage medium, wherein the size of the target space is greater than or equal to the size of the source storage space;
[0113] Step 3032: Determine the target address according to the address of the target space, the address of the target space is the destination storage address, and the target address has a one-to-one mapping relationship with the source address.
[0114] Exemplarily, after determining the source address, a target space can be applied for in the file system, and the target space points to the storage space in the storage medium. The target space can be continuous and of the same size as the source storage space of the to-be-processed data corresponding to the source address in the storage medium, or a continuous small-scale storage space whose sum is equal to the source storage space of the to-be-processed data corresponding to the source address in the storage medium, which is collectively referred to as the target address space set below. The target address is then determined according to the address of the target space, and the target address is the corresponding address in the file system.
[0115] Determining the target address in the above manner can ensure that the target address sufficiently corresponds to the source address, the address of the target space is the destination storage address, and the target address has a one-to-one mapping relationship with the source address, that is, when data is migrated, it is migrated from a source address to a corresponding target address. Among them, when the address in the source address is discontinuous, and the address in the determined target address is continuous, the data to be processed can be migrated from a discontinuous storage space to a continuous storage space. Among them, when the address in the source address is discontinuous, and the address in the determined target address is continuous, the data to be processed can be migrated from a discontinuous storage space to a continuous storage space, thereby realizing data organization in the storage space.
[0116] In some embodiments, the data to be processed may include multiple data, and when the data corresponds to different data processing methods, it is necessary to mark different marking information for each data and configure a target address for the data to be migrated.
[0117] In an embodiment of the present application, the data to be processed includes first data and second data, the data processing method corresponding to the first data is data migration, and the data processing method corresponding to the second data is data discard, the mark information corresponding to the data migration is a first mark, and the mark information corresponding to the data discard is a second mark, and determining the mark information corresponding to the data to be processed according to the data processing method may include: marking the mark information corresponding to the source address of the first data in the source address as a first mark, and marking the mark information corresponding to the source address of the second data in the source address as a second mark; determining the target address corresponding to the storage medium for migrating the data to be processed may include: determining the target address corresponding to the storage medium for migrating the first data.
[0118] It should be noted that when the data to be processed includes multiple data and corresponds to different data processing methods, using different marking information to mark them separately can facilitate the storage medium to perform corresponding data processing. For data whose data processing method is data migration, determine the corresponding target address.
[0119] It can be understood that the above embodiments respectively describe methods for obtaining the target address and tag information of the data to be processed. In some scenarios, obtaining the processing information requires obtaining the target address and tag information at the same time. The above methods for obtaining the target address and obtaining the tag information can be used in combination, which will not be repeated here.
[0120] Exemplarily, the file system can issue a certain continuous source address space, in which the data in the valid area will have a one-to-one correspondence with the target address space, and the invalid area can be marked by a special processing method agreed with the storage medium. When the address space is received by the storage medium, the storage medium can process different types of address spaces separately according to the screening results. For example, when the file system performs foreground garbage collection FGGC, the selected logical address space can be uniformly issued as a source address space set, and after the data in the valid source address space is migrated to the target address space, the entire logical source address space set can be uniformly released on the storage medium end.
[0121] In some embodiments, while processing the data, the storage location and / or storage method of the data may be further optimized according to the type of data.
[0122] Figure 7 A flow chart of another data processing method provided in an embodiment of the present application. Figure 7 As shown, the method includes:
[0123] Step 401: Obtain the source address and processing information of the data to be processed.
[0124] Step 402: Acquire type information of the data to be processed, where the type information is used to instruct the storage medium to determine a corresponding storage strategy according to the type information, where the storage strategy includes at least one of a storage method strategy and a storage location strategy.
[0125] Step 403: Generate an information packet according to the type information of the data to be processed, the source address and the processing information, and send the information packet to the storage medium.
[0126] It should be noted that step 401 may refer to the introduction in the aforementioned embodiment and will not be described in detail here.
[0127] In step 402, the type information of the data to be processed may be information that has been pre-generated by the file system, and the acquisition here may be reading the information. The type information of the data to be processed may also be information generated in the file system according to the data type of the data to be processed, and the acquisition here may be generating the information, etc. The embodiment of the present application does not limit the method for acquiring the type information of the data to be processed.
[0128] Among them, the type information of the data to be processed may be information obtained by classifying the data in different ways. Exemplarily, the file system may mark the data as hot data type and cold data type according to the frequency with which the data is copied. Hot data types refer to frequently used data, and cold data types refer to infrequently used data. The file system may also classify the data into types such as movie data, music data, and short video data. The embodiments of the present application do not limit the classification method of the data to be processed and the content of the type information.
[0129] In addition, the storage strategy may be a storage mode strategy and / or a storage area strategy. The storage mode strategy may be a method of selecting a sequential storage mode, a linked storage mode, an index storage mode, etc. according to different type information. The storage area strategy may be a method of selecting one of storage area A, storage area B, and storage area C according to different type information. Exemplarily, the storage mode strategy and the storage area strategy may also be used in combination, that is, the storage strategy may include different data storage modes and data storage areas corresponding to different type information.
[0130] In step 403, after the type information of the data to be processed is obtained, an information packet may be generated from the type information, the source address and the processing information, and sent to the storage medium.
[0131] It is understandable that if different types of data are mixed in the same storage array, it is obviously not conducive to garbage collection of the storage device. Therefore, the file system will classify the data and transmit the data type information, such as the shunt number, to the storage device when initiating an IO request to instruct the storage medium to determine and adopt the corresponding storage strategy to process the data to be processed according to the type information, in order to allow the storage device to make a management strategy that is more conducive to performance improvement.
[0132] Figure 8 A schematic diagram of another information packet structure provided in an embodiment of the present application. Figure 8 As shown in the figure, on the file system and storage medium side, you can customize a data packet structure for interaction. Figure 6 Based on the information packet shown, the information packet also includes type information of the data to be processed, and the type information is used to instruct the storage medium to determine the corresponding storage strategy according to the type information. In the process of data migration, the embodiment of the present application cooperates with the file system to adjust the data storage location to achieve the purpose of data classification / diversion storage.
[0133] In some embodiments, since storage media have significant differences in terms of process and performance, there needs to be clear limitations on whether different storage media support this solution, as well as the degree of data volume support, etc.
[0134] In an embodiment of the present application, before obtaining the source address and processing information of the data to be processed, the method may also include: obtaining a target parameter corresponding to the storage medium, the target parameter being used to indicate whether the storage medium supports a data processing function; and determining that a value of the target parameter is the same as a preset parameter value, the preset parameter value being a corresponding value when the storage medium supports the data processing function.
[0135] Exemplarily, taking mobile flash memory as an example, whether the storage medium supports the data processing function can be determined by the index number IDN and the device descriptor DEVICE DESCRIPTOR in the descriptor definition table of the storage medium.
[0136] Fig. 9 A schematic diagram of a descriptor definition table and a device descriptor provided in an embodiment of the present application. Fig. 9 As shown in the figure, the index number 50h in the left two columns of the descriptor definition table corresponds to the copy command COPY COMMAND, which represents the index number of the command in the software system. The offset value of 4Fh in the device descriptor indicates that it is an extended feature supported by the flash memory UFS. Its length is 4 bytes, and the bit Bit
[31] indicates whether the parameters of the data processing function are supported. When the value of Bit
[31] is 1, it means that the storage medium supports the copy command.
[0137] Fig.10 A schematic diagram of a parameter definition rule table for a data processing function provided in an embodiment of the present application. The parameter definition rule table includes detailed support parameters for the data processing function by the storage medium end, such as Fig.10 As shown in the figure, different offset values have corresponding parameter names, value ranges, and description information. The file system needs to comply with these when interacting with the storage medium. Fig.10 The rules shown, for example, generate information packets based on parameter definition rule tables, etc.
[0138] It is understandable that when the file system is started, it must first query whether the storage medium end supports the data processing solution. If not, the relevant path will be closed at the beginning of system initialization; otherwise, it is necessary to set the amount of data contained in the information packet according to the parameter definition rules during the execution of the data processing method, and the storage medium end also processes data according to the rules during the information packet format conversion process based on the parameter definition rules.
[0139] In some embodiments, after sending the information packet to the storage medium, the file system may wait for the processing result fed back by the storage medium to update the file system.
[0140] In an embodiment of the present application, the method may further include: receiving a processing result of the data to be processed sent by the storage medium; and maintaining information synchronization with the storage medium based on the processing result.
[0141] The data processing method provided in the embodiment of the present application decouples the two functions of data migration and address release on the file system and storage medium side, reduces the memory overhead and data input and output overhead of data processing, realizes fast data processing, and improves the working efficiency of the device.
[0142] Fig.11 The present invention provides another data processing method for implementing a flow chart, which is applied to a storage medium of an electronic device. Fig.11 As shown, the method may include the following steps 501 to 502:
[0143] Step 501: Obtain an information packet, the information packet including a source address of data to be processed and processing information, the source address is used to indicate a storage address of the data to be processed in the storage medium, and the processing information is used to instruct the storage medium to migrate and / or discard the data to be processed.
[0144] It should be noted that the storage medium may obtain the information packet by receiving the information packet or the like, and the embodiment of the present application does not limit the method for obtaining the information packet.
[0145] The information packet includes the source address and processing information of the data to be processed, that is, includes all the information required for the storage medium to process the data. Therefore, after the storage medium receives the information packet, it can directly process the data according to the information packet.
[0146] For example, when the information packet is sent to the driver layer of the flash memory, the driver layer can translate it into a format that the flash memory can parse according to the protocol. For example, if the mobile terminal uses the Small Computer System Interface SCSI protocol, the SCSI protocol write buffer mode can be used to convert the information packet into a format that the storage medium can parse. The command description block is as follows: Fig.12 As shown, the data content is as follows Fig.13 shown.
[0147] Fig.12 The data header format diagram of a command description block provided in an embodiment of the present application is a diagram of a command description block. The driver layer converts the information packet into a write buffer WRITE BUFFER command description block structure so that the storage medium can parse and execute it. Fig.10 As shown, the command description block includes parameters such as operation code, reserved bit, mode bit, cache number, cache offset, etc.
[0148] Fig.13A schematic diagram of the data content of a command description block provided in an embodiment of the present application, such as Fig.13 As shown, the data content includes operation code, function code, send length, reserved bit, number of blocks of block device, source logical block address of block device and destination logical block address of block device. Fig.12 and Fig.13 Together they constitute data processing commands that can be recognized by the storage medium.
[0149] It is understandable that after receiving the information packet, the driver layer Fig.12 and Fig.13 The data header structure and data content of the command description block in the command description block convert the information packet into a command description block that can be recognized by the storage medium, so that the storage medium controller can organize the data according to the command description block.
[0150] Step 502: Process the data to be processed according to the information packet.
[0151] It should be noted that the method for the storage medium to process data according to the information packet may be based on preset rules or other methods. The embodiment of the present application does not limit the method for processing the data to be processed according to the information packet.
[0152] It is understandable that the information packet obtained by the storage medium includes all the information required for data processing, so after the storage medium receives the information packet, it can directly process the data according to the information packet. This method decouples the storage medium's dependence on the file system during data processing, and the storage medium is completely responsible for the operation of obtaining the information packet and migrating or discarding the data according to the information packet, reducing the memory overhead and data input and output overhead of data processing, achieving fast data processing, and improving the working efficiency of the device.
[0153] In some embodiments, in step 101, the processing information can be divided into two modes according to the different information included in the processing information. The first mode may include: a target address, and the target address is used to instruct the storage medium to perform data migration and / or data discard on the data to be processed; the second mode may include: a target address and tag information, and the tag information is used to instruct the storage medium to perform data migration and / or data discard on the data to be processed, and the target address is used to indicate the destination storage address when the storage medium performs data migration on the data to be processed. The following is a detailed description based on the two modes.
[0154] In a first approach, the processing information includes a target address, and the target address is used to instruct the storage medium to perform data migration and / or data discard on the data to be processed.
[0155] It should be noted that different contents of the target address can be set to instruct the storage medium to migrate or discard the data to be processed. The target address can be obtained by the file system assigning a new storage address to the data to be processed or by other means. The embodiment of the present application does not limit the method of obtaining the target address.
[0156] Method 2: The processing information includes a target address and tag information, wherein the tag information is used to instruct the storage medium to migrate and / or discard the data to be processed, and the target address is used to indicate the destination storage address when the storage medium migrates the data to be processed.
[0157] It should be noted that different contents of the tag information can be set to instruct the storage medium to migrate or discard the data to be processed, and the target address is used to indicate the destination storage address of the data to be processed. The target address can be obtained by the file system assigning a new storage address to the data to be processed or by other methods, and the tag information can be obtained according to the correspondence between the preset processing method and the tag information. The embodiment of the present application does not limit the method of obtaining the target address and the tag information.
[0158] The following will describe the manner in which step 502 processes the data to be processed according to the information packet based on different processing information.
[0159] Processing method 1, the processing information includes the target address of the data to be processed in the storage medium, and the processing of the data to be processed according to the information packet may include: when the source address and the target address are different, migrating the data to be processed from the source address to the target address.
[0160] Processing method two: the processing information includes the target address of the data to be processed in the storage medium, and the processing of the data to be processed according to the information packet may include: when the source address is the same as the target address, discarding the data to be processed stored in the source address.
[0161] Wherein, in the case that the storage address includes multiple storage addresses and multiple target addresses, the multiple source addresses and the multiple target addresses have a one-to-one mapping relationship.
[0162] It should be noted that in Method 1 and Method 2, the processing information only includes the target address of the data to be processed in the storage medium. Therefore, data processing can be performed based on whether the storage address and the target address are the same. The above method is simple and easy to operate, which improves the efficiency of data processing.
[0163] In some embodiments, the information packet further includes type information of the data to be processed, and the storage medium may determine a storage strategy corresponding to the data to be processed according to the type information of the data to be processed.
[0164] Fig.14 A flowchart of a method for migrating data to be processed provided in an embodiment of the present application. Fig.14 As shown, the method includes:
[0165] Step 601: Determine a target storage strategy based on the type information of the data to be processed and a preset correspondence relationship, wherein the preset correspondence relationship is a correspondence relationship between the type information and the storage strategy, and the storage strategy includes at least one of a storage method strategy and a storage area strategy.
[0166] It should be noted that the information packet includes type information of the data to be processed, and the preset correspondence includes a correspondence between the type information and the storage strategy, so the target storage strategy corresponding to the type information of the data to be processed can be determined.
[0167] Exemplarily, the storage strategy may be a storage method strategy, and the storage method may be a sequential storage method, a linked storage method, an indexed storage method, etc. The storage method strategy may include different types of information corresponding to different storage methods, that is, the preset correspondence relationship may be that different types of information correspond to different storage methods.
[0168] Exemplarily, the storage strategy may be a storage area strategy, and the storage areas may include storage area A, storage area B, and storage area C, etc. The storage area strategy may include different type information corresponding to different storage areas, that is, the preset correspondence relationship is that different type information corresponds to different storage areas.
[0169] Exemplarily, the storage mode strategy and the storage area strategy may also be used in combination, that is, the preset correspondence may include different types of information corresponding to different data storage modes and data storage areas.
[0170] Step 602: Migrate the data to be processed from the source address to the target address according to the target storage policy.
[0171] It should be noted that after determining the target storage strategy, the data to be processed can be migrated from the source address to the target address according to the target storage strategy. Exemplarily, the corresponding storage method can be used to store the data in the corresponding storage area. The embodiment of the present application does not limit the method of migrating data. Among them, the target address refers to the logical address, and the storage area allocated by the target storage strategy for the data to be processed is the physical address for storing the data to be processed.
[0172] It is understandable that the embodiment of the present application stores the same type of data, such as data with the same stream number, together, so as to achieve the purpose of data shunting storage and wear leveling. However, the type information of the data, such as the shunting number, is not static and may be converted to each other. In the embodiment of the present application, according to the type information of the move command issued, the storage method and storage location of the data in the storage array can be adjusted while the file is being sorted, so as to improve the performance of the entire system under limited resources.
[0173] In some embodiments, after the information packet is sent to the driver layer of the flash memory, the driver layer can translate it into a format that the flash memory can parse according to the protocol. For example, if the mobile terminal uses the SCSI protocol, then an operation code Opcode, such as 0xCC, can be newly defined in the UFS SCSI Command Set or the UFS Native Command Set. The structure of the command description block is as follows Figure 15-17 shown.
[0174] Fig.15 A schematic diagram of the data header format of a command description block provided in an embodiment of the present application. Fig.15 As shown in FIG. 1 , the data header of the command description block includes parameters such as the command code, reserved bits, the destination logical address of the block device specified by the command, the number of blocks in the block device, the total length of the parameter table, and the control code. Among them, the MSB (Most Significant Bit) represents the most significant bit, and the LSB (Least Significant Bit) represents the least significant bit.
[0175] Fig.16 A schematic diagram of the format definition of a copy command parameter table of a command description block provided in an embodiment of the present application. Fig.16 As shown, the copy command parameter table includes the descriptor from the 0th device block to the descriptor of the nth device block.
[0176] Fig.17 The following is a schematic diagram of a format definition of a device block descriptor provided in an embodiment of the present application. Fig.14 As shown, the descriptor of the device block includes parameters such as the source logical block address in the source device, the number of blocks in the block device, and the stream number specified by the command to be moved.
[0177] in, Fig.15 , Fig.16 and Fig.17 Together they constitute data processing commands that can be recognized by the storage medium.
[0178] In some embodiments, the driver layer may also define another CDB structure in the UFS SCSI Command Set or UFS Native Command Set. Figure 15-17 shown.
[0179] Fig.18 A schematic diagram of a data header format of another command description block provided in an embodiment of the present application. Fig.18 As shown, the data header of the command description block includes parameters such as the command code, reserved bits, the destination logical address of the block device specified by the command, the number of blocks in the block device, the stream number specified by the command to be moved, and the total length of the parameter table.
[0180] Fig.19 A schematic diagram of a format definition of a copy command parameter table of another command description block provided in an embodiment of the present application. Fig.19 As shown, the data content of the copy command parameter table includes the descriptor from the 0th device block to the descriptor of the nth device block.
[0181] Fig. 20 A schematic diagram of a format definition of a descriptor of another device block provided in an embodiment of the present application. Fig. 20 As shown, the descriptor of the device block includes parameters such as the source logical block address in the block device, the number of blocks in the block device, and the reserved bit.
[0182] in, Fig.18 , Fig.19 and Fig. 20 Together they constitute data processing commands that can be recognized by the storage medium.
[0183] Fig.21 A schematic diagram of a flow chart of data processing by a storage medium provided in an embodiment of the present application, such as Fig.21 As shown, after the storage medium receives the information packet, the storage medium controller parses it to obtain the source address space set and the target address space set. The controller then searches for the data stored in the source address through the source address space set and migrates the data to the address of the target address space set, thereby realizing the migration of the data to be processed. This method reduces the overhead of data reading and storage and improves the efficiency of data migration.
[0184] Processing method three, the processing information includes the target address and the tag information, and the processing of the data to be processed according to the information packet may include: when the data processing method indicated by the tag information is data migration, migrating the data to be processed from the source address to the target address; when the data processing method indicated by the tag information is data discarding, discarding the data to be processed stored in the source address.
[0185] It should be noted that, when the processing information includes the target address and the tag information, the data to be processed can be either migrated or discarded. When the data processing method is determined to be data migration, the target address can be used for data migration. When the data processing method is determined to be data discard, the target address is not needed. When the address in the source address is discontinuous and the address in the target address is continuous, the data to be processed is migrated from the discontinuous storage space to the continuous storage space for data sorting.
[0186] It is understandable that the third method can process data in the space to be processed in the storage medium, for example, migrate valid data in the space to be processed and discard invalid data in the space to be processed.
[0187] Furthermore, the data to be processed is valid data in a storage space to be processed in the storage medium. After the data to be processed is processed according to the information packet, the method may further include: releasing all storage space in the storage space to be processed.
[0188] It should be noted that after data processing is performed on the data in the to-be-processed space in the storage medium, for example, after valid data is migrated, all storage space of the to-be-processed space may be released.
[0189] Fig. 22 A schematic diagram of a data processing flow provided in an embodiment of the present application, such as Fig. 22 As shown, in the application scenario of data migration of the embodiment of the present application, in the process of processing data of the processing space or a single file, not only the efficiency of data migration is improved, but also the space release discard / trim strategy is integrated into it, so as to reduce the number of times such commands are issued, reduce the burden on the system and achieve twice the result with half the effort. The embodiment of the present application achieves the effect of one instruction covering multiple functions by integrating the space release discard / trim function.
[0190] Fig.23 A schematic diagram of a method for processing data in a to-be-reclaimed area provided in an embodiment of the present application is shown in FIG. Fig.23 As shown in the figure, V represents Valid, and I represents Invalid, an invalid area. The storage medium can directly migrate the data pointed to by the source address space of the valid area in the area to be recovered to the target address space according to the issued address. Then, when all the data is moved, the entire source address space can be recovered and released on the file system side and the storage medium side. In this way, the data can be quickly migrated and a continuous space address can be recovered through one IO instruction.
[0191] In some embodiments, the method may further include: sending the processing result of the to-be-processed data to a file system, so that the file system and the storage medium maintain information synchronization.
[0192] It should be noted that since the processing speed of the storage medium controller is far lower than that of the system processor, it is not necessary to wait until the data is written to the non-volatile storage medium before returning a response immediately. However, for important or necessary data, it is also possible to return the processing results after the data processing is completed.
[0193] The data processing method provided in the embodiment of the present application disassembles and redistributes the workload of data processing in the native data processing system, realizes data processing through asynchronous operations of storage media and file systems, and improves the activity of the device while fully reducing the system load.
[0194] The following will describe an exemplary application of the embodiment of the present application in a practical application scenario. The file system in the example adopts a log structure file system.
[0195] In a log-structured file system, invalid data is inevitably generated due to the use of a remotely updated data writing method. However, in the early stages of system operation, the presence of these invalid data may not have a significant impact on performance, but as these invalid data accumulate over time, they will lead to a shortage of system space. The obvious situation is that in theory the system space is sufficient, but new data has nowhere to go, so it is imperative to recycle these spaces reasonably and quickly. In combination with the technical solution provided in the embodiments of the present application, the rapid migration of valid data can be achieved, thereby quickly providing free space for the system.
[0196] Fig.24 The following is a schematic diagram of the overall flow of a data processing method provided in an embodiment of the present application. Fig.24As shown, the method includes: 1) the file system determines the source address space set. Among them, since the file system adopts a log structure file system, first, for the data to be sorted, the address storing these data is found on the file system side, which is collectively referred to as the source address below. At this time, these source addresses should be partially continuous and partially non-continuous. After finding these addresses, they are saved in a temporary storage space, which is collectively referred to as the source address space set below. 2) The file system determines the target address space set. According to the characteristics of off-site update, a continuous small-scale storage space with a size equal to the storage space size corresponding to the source address space set or a sum of several blocks equal to the storage space size corresponding to the source address space set is applied for in the file system as the target, which is collectively referred to as the target address space set below. 3) The file system determines the address in the source address space set and the address in the corresponding target address space set. Among them, the addresses in the source address space set are divided according to the management granularity of the file system, and the corresponding addresses in the target address space set are determined respectively. 4) The file system generates information packets from the source address space set and the corresponding target address space set and sends them down. 5) The storage medium performs data migration according to the received information packets. Among them, after receiving the information packet, the storage medium can choose to reply to the software end, namely the file system, and is responsible for moving the data from the source address in the source address space set to the address space specified by the target address space set one by one.
[0197] It is understandable that after the storage medium receives the information packet, if the address stored in the source address space set in the information packet is different from the address stored in the target address space set, it means that this is a data migration instruction. On the contrary, if the address spaces stored in the two are the same, it means that this is a data discard instruction, and the data corresponding to the address in the source address space set needs to be discarded.
[0198] In the process of using the log structure file system, operations such as adding, deleting, modifying and checking files are inevitable. With the frequent update of files, the file storage form will become increasingly fragmented. If a large number of files stored in the system are in this discrete state, then the expected result is that the system will be stuck and the user experience will deteriorate. Therefore, in order to avoid this negative effect, it is imperative to organize the files. As we all know, the speed of file organization is inversely proportional to the size of the file, so it is a luxury to be able to complete as many file organization as possible within a unit of time.
[0199] Fig.25 A schematic diagram of a data processing method based on a target file provided in an embodiment of the present application, such as Fig.25As shown, V represents valid data of the target file, O represents valid data of other files, I represents invalid data, and F represents a blank data block. After receiving the data sorting command for the target file, the file system starts to execute the instruction, and will search for the source storage information of the sorted target file from the area to be recovered. The light-colored marks are valid data, and it can be seen that the valid data is dispersed. The addresses of the valid data are counted in the source address space, and the target address space can be a continuous space. The source address set and the target address set are packaged and sent to the storage medium, and the storage medium realizes data migration. The method can fully reduce the IO overhead of the target file during the data migration process, and greatly improve the efficiency of data migration and shorten the time consumption.
[0200] The embodiment of the present application combines the characteristics of off-site update of log-structured file systems, and proposes a data processing method with the core idea of reducing costs and increasing efficiency, where each performs its own duties. The so-called cost reduction and efficiency increase means to minimize various expenses, including the number of input and output IO and the effective use frequency of page cache memory page cache. The so-called performing its own duties means to give full play to the asynchronous advantages of software (file system) and devices (storage medium), fully call the initiative of devices, and reduce software interference. In this way, software resources can be quickly released and work efficiency can be improved.
[0201] Fig.26 A schematic diagram of a scenario of a data processing method based on type information provided in an embodiment of the present application. Fig.26 As shown, V represents valid data of one file, J represents valid data of another file, I represents invalid data, and F represents a blank data block. The type information of the data corresponding to file V is different from the type information of the data corresponding to file J, such as the shunt number, that is, the shunt number corresponding to file V is a, and the shunt number corresponding to file J is b. The file system will search and organize the source storage information of file V and file J from the areas to be organized 1-3. It can be seen that the valid data of file V and the valid data of file J are dispersed. The addresses of valid data are counted in the source address space, and the target address space can be a continuous space. The type information (shunt number), source address set and target address set of the data of file V and file J are packaged and sent to the storage medium. As can be seen from the figure, the storage medium will move the data to the corresponding storage area through the ath data stream according to the shunt number a of file V, and move the data to the corresponding storage area through the bth data stream according to the shunt number b of file J, that is, the data of file V and file J are classified and stored. The method adjusts the storage mode and storage position of data in the storage array while arranging files, thereby improving the performance of the entire system under limited resources.
[0202] It should be understood that, although the various steps in the above-mentioned flowchart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the above-mentioned flowchart may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.
[0203] Based on the foregoing embodiments, the embodiments of the present application provide a data processing device, and the modules included in the device and the units included in the modules can be implemented by a processor; of course, they can also be implemented by a specific logic circuit; in the implementation process, the processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP) or a field programmable gate array (FPGA), etc.
[0204] Fig. 27 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application is shown in FIG. Fig. 27 As shown, the device 700 includes a first acquisition module 701 and an information sending module 702, wherein:
[0205] A first acquisition module 701 is used to acquire a source address and processing information of the data to be processed, wherein the source address is used to indicate a storage address of the data to be processed in a storage medium, and the processing information is used to instruct the storage medium to perform data migration and / or data discard on the data to be processed;
[0206] The information sending module 702 is used to generate an information packet according to the source address and the processing information and send the information packet to the storage medium, so that the storage medium processes the data to be processed according to the information packet.
[0207] In some embodiments, the processing information includes the target address of the data to be processed on the storage medium, and / or, tag information corresponding to the data to be processed, the target address indicates the target address for data migration or data discarding, and the tag information indicates that the data processing method is data migration or data discarding.
[0208] In some embodiments, the processing information includes the target address of the data to be processed in the storage medium, and the first acquisition module 701 includes a space determination unit and an address determination unit, wherein:
[0209] a space determination unit, configured to determine a target space in the storage medium according to a size of a source storage space of the data to be processed corresponding to the source address in the storage medium, wherein the size of the target space is greater than or equal to the size of the storage space corresponding to the source address;
[0210] The address determining unit is used to determine the address corresponding to the target space as the target address.
[0211] In some embodiments, the processing information includes tag information corresponding to the data to be processed, and the first acquisition module 701 includes a mode determination unit, a tag determination unit and a tag operation unit, wherein:
[0212] A method determining unit, used to determine a data processing method corresponding to the data to be processed;
[0213] a tag determination unit, configured to obtain target tag information corresponding to the target data processing method according to a preset corresponding relationship and the target data processing method, wherein the preset corresponding relationship is a corresponding relationship between the data processing method and the tag information;
[0214] The marking operation unit is used to mark the marking information corresponding to the source address as target marking information, so as to obtain the marking information corresponding to the data to be processed.
[0215] In some embodiments, the first acquisition module 701 includes an address acquisition unit, which is used to acquire the source address from a preset storage space of the file system, and the source address is stored in the preset storage space after the data to be processed is found.
[0216] In some embodiments, the data processing device 700 also includes: a function determination module, used to obtain a target parameter corresponding to the storage medium, the target parameter is used to indicate whether the storage medium supports the data processing function; determine that the value of the target parameter is the same as a preset parameter value, and the preset parameter value is the corresponding value when the storage medium supports the data processing function.
[0217] In some embodiments, the data processing device 700 further includes: a system update module, configured to receive a processing result of the data to be processed sent by the storage medium; and to synchronize information with the storage medium according to the processing result.
[0218] In some embodiments, the data processing device 700 further includes: a type acquisition module, configured to acquire type information of the data to be processed, wherein the type information is used to instruct the storage medium to determine a corresponding storage strategy according to the type information, wherein the storage strategy includes at least one of a storage method strategy and a storage location strategy;
[0219] The information sending module 702 is further used to generate an information packet according to the type information of the data to be processed, the source address and the processing information, and send the information packet to the storage medium.
[0220] Fig.28 A structural diagram of another data processing device provided in an embodiment of the present application is shown in FIG. Fig.28 As shown, the device 800 includes a second acquisition module 801 and a data processing module 802, wherein:
[0221] The second acquisition module 801 is used to acquire an information packet, the information packet including a source address of the data to be processed and processing information, the source address is used to indicate a storage address of the data to be processed in a storage medium, and the processing information is used to instruct the storage medium to perform data migration and / or data discard on the data to be processed;
[0222] The data processing module 802 is used to process the data to be processed according to the information packet.
[0223] In some embodiments, the processing information includes the target address of the data to be processed on the storage medium, and / or, tag information corresponding to the data to be processed, the target address indicates the target address for data migration or data discarding, and the tag information indicates that the data processing method is data migration or data discarding.
[0224] In some embodiments, the processing information includes the target address, and the data processing module 802 is specifically used to: when the source address is different from the target address, migrate the to-be-processed data from the source address to the target address.
[0225] In some embodiments, the processing information includes the target address, and the data processing module 802 is specifically used to: when the source address is the same as the target address, discard the to-be-processed data stored in the source address.
[0226] In some embodiments, the processing information includes the tag information, and the data processing module 802 is specifically used to discard the to-be-processed data stored in the source address when the data processing mode indicated by the tag information is data discard.
[0227] In some embodiments, the processing information includes the target address and the tag information, and the data processing module 802 is specifically used to:
[0228] When the data processing mode indicated by the tag information is data migration, migrating the data to be processed from the source address to the target address;
[0229] When the data processing mode indicated by the mark information is data discard, the to-be-processed data stored in the source address is discarded.
[0230] In some embodiments, the data to be processed is valid data in the space to be processed, the space to be processed is storage space in the storage medium, and the data processing device 800 also includes a space release module for releasing all storage space in the space to be processed.
[0231] In some embodiments, the data processing device 800 further includes a result sending module, which is used to send the processing result of the to-be-processed data to a file system, so that the file system and the storage medium can keep information synchronized.
[0232] In some embodiments, the data processing device 800 further includes a policy acquisition module, which is used to determine a target storage policy according to the type information of the data to be processed and a preset corresponding relationship, wherein the preset corresponding relationship is a corresponding relationship between the type information and the storage policy, and the storage policy includes at least one of a storage mode policy and a storage area policy;
[0233] The data processing module 802 is further configured to: migrate the to-be-processed data from the source address to the target address according to the target storage strategy.
[0234] In the embodiment of the present application, the processing of the internal data of the storage medium can be completed through the asynchronous operation of the file system and the storage medium, thereby reducing the memory overhead and data input and output overhead of data processing, achieving fast data processing, and improving the working efficiency of the device.
[0235] The description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.
[0236] It should be noted that in the embodiments of this application Fig. 27 and Fig.28 The division of modules in the data processing device shown is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional unit in each embodiment of the present application may be integrated into a processing unit, or may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of a software functional unit. It may also be implemented in the form of a combination of software and hardware.
[0237] It should be noted that in the embodiment of the present application, if the above method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application can be essentially or partly embodied in the form of a software product that contributes to the relevant technology. The computer software product is stored in a storage medium, including several instructions to enable an electronic device to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific combination of hardware and software.
[0238] Fig.29 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Fig.29 As shown, the electronic device 100 may include a processor 110 and a memory 120 , among others.
[0239] The processor 110 may include one or more processing units. For example, the processor 110 is a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (FPGAs). Different processing units may be independent devices or integrated into one or more processors.
[0240] The memory 120 can be used to store computer executable program codes, which include instructions. The internal memory 120 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, video data, etc.), etc. In addition, the memory 120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the memory 120, and / or instructions stored in a memory provided in the processor.
[0241] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.
[0242] An embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any one of the data processing methods provided in the above embodiments are implemented.
[0243] The above-mentioned computer-readable storage medium can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM) or flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.
[0244] Computer-readable signal media may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.
[0245] The program code embodied on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wire, optical cable, radio frequency (RF), etc., or any suitable combination of the foregoing.
[0246] Computer program code for performing the operations of the present specification may be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0247] An embodiment of the present application provides a computer program product including instructions, which, when executed on a computer, enables the computer to execute the steps of any one of the data processing methods provided in the above method embodiments.
[0248] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a server, or a data center to another website site, a computer, a server, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)), etc.
[0249] It should be noted here that the description of the above storage medium and program product embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium, program product, and device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.
[0250] It should be understood that "one embodiment" or "an embodiment" or "some embodiments" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in one embodiment" or "in some embodiments" appearing throughout the specification may not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application. The above-mentioned sequence numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments. The above description of each embodiment tends to emphasize the differences between the various embodiments, and the same or similar aspects can be referenced to each other. For the sake of brevity, this article will not repeat them.
[0251] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, object A and / or object B can represent three situations: object A exists alone, object A and object B exist at the same time, and object B exists alone.
[0252] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.
[0253] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as: multiple modules or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or modules can be electrical, mechanical or other forms.
[0254] The modules described above as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules; they may be located in one place or distributed on multiple network units; some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.
[0255] In addition, all functional modules in the embodiments of the present application may be integrated into one processing unit, or each module may be a separate unit, or two or more modules may be integrated into one unit; the above-mentioned integrated modules may be implemented in the form of hardware or in the form of hardware plus software functional units.
[0256] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), magnetic disks or optical disks, etc., various media that can store program codes.
[0257] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application can essentially or in other words, the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling an electronic device to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.
[0258] The methods disclosed in several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.
[0259] The features disclosed in several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.
[0260] The features disclosed in several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.
[0261] The above is only an implementation method of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A data processing method, It is characterized in that Applied to a file system, the method comprises: Acquire a source address and processing information of the data to be processed, wherein the source address is used to indicate a storage address of the data to be processed in a storage medium, and the processing information is used to instruct the storage medium to perform data migration and / or data discard on the data to be processed; An information packet is generated according to the source address and the processing information and sent to the storage medium, so that the storage medium processes the data to be processed according to the information packet.
2. The method according to claim 1, It is characterized in that The processing information includes a target address, and the target address is used to instruct the storage medium to perform data migration and / or data discard on the data to be processed.
3. The method according to claim 2, It is characterized in that The obtaining of processing information of the data to be processed includes: Determine a data processing method corresponding to the data to be processed; If the data processing mode is data discard, determining the target address to be the source address; If the data processing method is data migration, it is determined that the target address corresponds to a destination storage address of the storage medium to migrate the data to be processed.
4. The method according to claim 3, It is characterized in that Determining that the target address corresponds to the destination storage address of the storage medium for migrating the data to be processed includes: Determine a target space in the storage medium according to the size of the source storage space of the data to be processed corresponding to the source address in the storage medium, wherein the size of the target space is greater than or equal to the size of the source storage space; The target address is determined according to the address of the target space, the address of the target space is the destination storage address, and the target address has a one-to-one mapping relationship with the source address.
5. The method according to claim 4, It is characterized in that The determining the target space in the storage medium according to the size of the source storage space of the data to be processed corresponding to the source address in the storage medium includes: Determine the storage space in the storage medium corresponding to the target log header in the file system as the target storage space, wherein the storage space in the storage medium corresponding to the target log header is a preset storage space for storing the data to be processed; A storage space having a size greater than or equal to that of the source storage space is determined from the target storage space as the target space.
6. The method according to claim 4 or 5, It is characterized in that The target space includes at least two target subspaces, the size of each target subspace is equal to or less than a space allocation threshold, the space allocation threshold is a maximum storage space in the storage medium allocated by the file system at one time, and determining the target address according to the address of the target space includes: The target address is determined according to the addresses of the at least two target subspaces.
7. The method according to claim 6, It is characterized in that The step of generating an information packet according to the source address and the processing information and sending the information packet to the storage medium comprises: Generate at least two sub-information packets according to the source address and the target address, each of the sub-information packets includes a target number of the target addresses and a target number of the source addresses, the target number being less than or equal to the number of addresses corresponding to one target subspace; Each of the at least two sub-information packets is sent to the storage medium.
8. The method according to claim 4, It is characterized in that The data to be processed includes valid data and invalid data, and determining the target space in the storage medium according to the size of the source storage space of the data to be processed corresponding to the source address in the storage medium includes: The target space is determined according to the size of the storage space of the valid data in the storage medium corresponding to the source address of the valid data in the source address.
9. The method according to claim 1, It is characterized in that The processing information includes a target address and tag information, wherein the tag information is used to instruct the storage medium to perform data migration and / or data discard on the data to be processed, and the target address is used to indicate the destination storage address when the storage medium performs data migration on the data to be processed.
10. The method according to claim 9, It is characterized in that The obtaining of processing information of the data to be processed includes: Determine a data processing method corresponding to the data to be processed; Determine, according to the data processing method, the tag information corresponding to the data to be processed, wherein the data processing method and the tag information have a corresponding relationship; When the data processing mode is data migration, it is determined that the target address corresponds to a destination storage address of the storage medium for migrating the data to be processed.
11. The method according to claim 10, It is characterized in that Determining that the target address corresponds to the destination storage address of the storage medium for migrating the data to be processed includes: Determine a target space in the storage medium according to the size of the source storage space of the data to be processed corresponding to the source address in the storage medium, wherein the size of the target space is greater than or equal to the size of the source storage space; The target address is determined according to the address of the target space, the address of the target space is the destination storage address, and the target address has a one-to-one mapping relationship with the source address.
12. The method according to claim 10, It is characterized in that The data to be processed includes first data and second data, the data processing mode corresponding to the first data is data migration, the data processing mode corresponding to the second data is data discarding, the tag information corresponding to the data migration is a first tag, and the tag information corresponding to the data discarding is a second tag, and determining the tag information corresponding to the data to be processed according to the data processing mode includes: Marking the tag information corresponding to the source address of the first data in the source address as a first tag, and marking the tag information corresponding to the source address of the second data in the source address as a second tag; Determining that the target address corresponds to the destination storage address of the storage medium for migrating the data to be processed includes: It is determined that the target address corresponds to a destination storage address of the storage medium for migrating the first data.
13. The method according to any one of claims 1-5, 7-12, It is characterized in that The obtaining the source address of the data to be processed includes: The source address is obtained from a preset storage space of the file system, and the source address is stored in the preset storage space after the data to be processed is found.
14. The method according to claim 1, It is characterized in that Before obtaining the source address and processing information of the data to be processed, the method further includes: Acquire a target parameter corresponding to the storage medium, where the target parameter is used to indicate whether the storage medium supports a data processing function; It is determined that the value of the target parameter is the same as the preset parameter value, and the preset parameter value is the corresponding value when the storage medium supports the data processing function.
15. The method according to claim 4 or 11, It is characterized in that The addresses in the source address are discontinuous, and the addresses in the determined target address are continuous.
16. The method according to claim 1, It is characterized in that The method further comprises: Receiving a processing result of the data to be processed sent by the storage medium; Information is synchronized with the storage medium according to the processing result.
17. The method according to claim 1, It is characterized in that Before generating an information packet according to the source address and the processing information and sending the information packet to the storage medium, the method includes: Acquire type information of the data to be processed, where the type information is used to instruct the storage medium to determine a corresponding storage strategy according to the type information, where the storage strategy includes at least one of a storage method strategy and a storage location strategy; The step of generating an information packet according to the source address and the processing information and sending the information packet to the storage medium comprises: An information packet is generated according to the type information of the data to be processed, the source address and the processing information, and is sent to the storage medium.
18. A data processing method, It is characterized in that Applied to a storage medium, the method comprises: Acquire an information packet, the information packet including a source address of the data to be processed and processing information, the source address is used to indicate a storage address of the data to be processed in a storage medium, and the processing information is used to instruct the storage medium to perform data migration and / or data discard on the data to be processed; The data to be processed is processed according to the information packet.
19. The method according to claim 18, It is characterized in that The processing information includes a target address, and the target address is used to instruct the storage medium to perform data migration and / or data discard on the data to be processed.
20. The method according to claim 19, It is characterized in that The processing of the data to be processed according to the information packet includes: In a case where the source address is different from the target address, the data to be processed is migrated from the source address to the target address.
21. The method according to claim 19, It is characterized in that The processing of the data to be processed according to the information packet includes: In the case where the source address is the same as the target address, the to-be-processed data stored in the source address is discarded.
22. The method according to claim 18, It is characterized in that The processing information includes a target address and tag information, wherein the tag information is used to instruct the storage medium to perform data migration and / or data discard on the data to be processed, and the target address is used to indicate the destination storage address when the storage medium performs data migration on the data to be processed.
23. The method according to claim 22, It is characterized in that The processing of the data to be processed according to the information packet includes: When the data processing mode indicated by the tag information is data migration, migrating the data to be processed from the source address to the target address; When the data processing mode indicated by the mark information is data discard, the to-be-processed data stored in the source address is discarded.
24. The method according to claim 20 or 23, It is characterized in that The information packet further includes type information of the data to be processed. Before migrating the data to be processed from the source address to the target address, the method further includes: Determine a target storage strategy according to the type information of the data to be processed and a preset corresponding relationship, wherein the preset corresponding relationship is a corresponding relationship between the type information and the storage strategy, and the storage strategy includes at least one of a storage mode strategy and a storage area strategy; The migrating the to-be-processed data from the source address to the target address includes: According to the target storage policy, the data to be processed is migrated from the source address to the target address.
25. The method according to claim 18, It is characterized in that The data to be processed is valid data in the storage space to be processed in the storage medium. After the data to be processed is processed according to the information packet, the method further includes: All storage spaces in the storage space to be processed are released.
26. The method according to claim 18, It is characterized in that The information packet includes a plurality of the source addresses and a plurality of the target addresses, and the plurality of the source addresses and the plurality of the target addresses have a one-to-one mapping relationship.
27. The method according to claim 19 or 22, It is characterized in that The addresses in the source address are discontinuous, and the addresses in the target address are continuous.
28. The method according to claim 18, It is characterized in that The method further comprises: The processing result of the data to be processed is sent to a file system, so that the file system and the storage medium keep information synchronization.
29. A data processing device, It is characterized in that Used in electronic equipment, including: A first acquisition module, used to acquire a source address and processing information of the data to be processed, wherein the source address is used to indicate a storage address of the data to be processed in a storage medium, and the processing information is used to instruct the storage medium to perform data migration and / or data discard on the data to be processed; The information sending module is used to generate an information packet according to the source address and the processing information and send the information packet to the storage medium, so that the storage medium processes the data to be processed according to the information packet.
30. A data processing device, It is characterized in that Used in electronic equipment, including: A second acquisition module is used to acquire an information packet, wherein the information packet includes a source address of the data to be processed and processing information, wherein the source address is used to indicate a storage address of the data to be processed in a storage medium, and the processing information is used to instruct the storage medium to perform data migration and / or data discard on the data to be processed; A data processing module is used to process the data to be processed according to the information packet.
31. An electronic device comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, It is characterized in that When the processor executes the program, it implements the steps of the data processing method described in any one of claims 1 to 17, or the steps of the data processing method described in any one of claims 18 to 28.
32. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the computer program implements the steps of the data processing method according to any one of claims 1 to 17, or the steps of the data processing method according to any one of claims 18 to 28.
Citation Information
Patent Citations
Data migration sending method and device, storage medium and electronic device
CN112650710A
Inter-process data migration method and device
CN113407357A
Data migration method, terminal, storage medium and program product
CN115080505A
Hardware based memory migration and resilvering
US20130268739A1
Storage system for supporting copy command and move command and operation method of storage system
US20140068216A1
Cited By
Memory control method, storage device, medium and computer equipment
CN121411709A
A memory control method, a storage device, a medium and a computer device
CN121411709B
