An object processing method, apparatus, electronic device, and storage medium

Abstract

This application provides an object processing method, apparatus, electronic device, and storage medium. The method includes: acquiring a data operation request sent by a target device; parsing index information from the data operation request, the index information including object size and storage location; selecting a target storage pool from multiple storage pools based on the object size, wherein at least two of the multiple storage pools have different maximum object sizes; retrieving a target object from the target storage pool based on its storage location; and performing corresponding operations on the target object according to the data operation request. By setting the maximum object size to be different in at least two of the multiple storage pools, even if any object data in the multiple storage pools is deleted, the idle space after the deletion can be reused by other objects. This avoids the impact of re-merging on the target object and improves the timeliness of responding to the target object's operation.

Description

Technical Field

[0001] This application relates to the technical field of computer storage and object storage, and more specifically, to an object processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] Currently, in the field of object storage device (OSD) technology, OSD servers need to reduce the number of write operations by appending small files from object storage to aggregate files. Specifically, when the object data in a small file is less than or equal to a preset threshold for the first write operation, it is appended to the aggregate file, and the identifier of the aggregate file, the length of the object data, and the storage location are recorded in the metadata table. Then, the aggregate file is only written to the OSD server when its capacity reaches the set capacity, thereby reducing the number of write operations on the OSD server.

[0003] However, because there is a lot of free space after object data is deleted from the aggregate file, the OSD server spends most of its time re-merging this free space, reducing the server's available computing resources. If a query request is sent to this OSD server, the target object being queried may be occupied during the merging process, and with reduced available computing resources, it is difficult to receive a timely response. Therefore, the current OSD server has low timeliness in responding to target object operations. Summary of the Invention

[0004] The purpose of this application is to provide an object processing method, apparatus, electronic device, and storage medium to improve the problem of low timeliness in responding to target objects.

[0005] This application provides an object processing method, including: acquiring a data operation request sent by a target device; parsing index information from the data operation request, the index information including: object size and storage location; selecting a target storage pool from multiple storage pools based on the object size, wherein at least two of the multiple storage pools have different maximum object sizes; acquiring a target object from the target storage pool based on the storage location; and performing corresponding operations on the target object according to the data operation request. In the implementation of the above scheme, since at least two of the multiple storage pools have different maximum object sizes, even if any object data in multiple storage pools is deleted, the idle space after the deletion can be reused by other objects. That is, the idle space can be stored and occupied by target objects smaller than the maximum object size of the storage pool, thus avoiding the impact of re-merging on the target object and improving the timeliness of responding to the target object's operation.

[0006] Optionally, in this embodiment, parsing index information from a data operation request includes: parsing a file name from the data operation request; and searching for the index information corresponding to the file name in a database, where the database stores the correspondence between file names and index information. In the implementation of the above scheme, by parsing the file name from the data operation request and searching for the corresponding index information in the database, the target device can operate on the object simply by its file name, effectively improving the ease of operation of the target object.

[0007] Optionally, in this embodiment, the multiple storage pools include: a first storage pool and a second storage pool, wherein the maximum object size stored in the first storage pool is smaller than the maximum object size stored in the second storage pool; selecting a target storage pool from the multiple storage pools based on the object size includes: determining whether the object size meets preset conditions, the preset conditions including: the object size is greater than the maximum object size stored in the first storage pool and the object size is smaller than the maximum object size stored in the second storage pool; if so, the second storage pool is determined as the target storage pool; otherwise, the first storage pool is determined as the target storage pool. In the implementation of the above scheme, since the maximum object size stored in the first storage pool is smaller than the maximum object size stored in the second storage pool, even if the object data in the first or second storage pool is deleted, the idle space in the first or second storage pool can be reused by other objects, thus avoiding the impact of re-merging on the target object and improving the timeliness of responding to the target object.

[0008] Optionally, in this embodiment, the storage pool includes: multiple files, each file including multiple file segments, the size of which is equal to the maximum object size stored in the target storage pool; retrieving the target object from the target storage pool based on its storage location includes: locating the target file containing the target object from the multiple files in the target storage pool based on its storage location; locating the target file segment from the multiple file segments in the target file based on its storage location; and reading the target object from the target file segment. In the implementation of the above scheme, by storing objects using multiple file segments included in the files in the storage pool, the total number of files in the file system is reduced, and the disk space utilization and read / write performance of the storage engine are improved.

[0009] Optionally, in this embodiment, the data operation request is a query request; performing corresponding operations on the target object according to the data operation request includes: generating a query response corresponding to the query request based on index information and the target object, and sending the query response corresponding to the query request to the target device. In the implementation of the above scheme, by generating a query response corresponding to the query request based on index information and the target object, and sending the query response corresponding to the query request to the target device, the impact of re-merging on the operation of the target object is avoided, thereby improving the timeliness of responding to the target object's operations.

[0010] Optionally, in this embodiment, the data operation request is a deletion request; performing corresponding operations on the target object according to the data operation request includes deleting the target object from the target storage pool. In the implementation of the above scheme, deleting the target object from the target storage pool avoids the impact of re-merging on the target object, thereby improving the timeliness of responding to the target object's operations.

[0011] Optionally, in this embodiment, the method further includes: receiving a storage request sent by the target device, and parsing the object to be processed and its size from the storage request; selecting a target storage pool from multiple storage pools based on the object size, and storing the object to be processed in the storage location of the target storage pool, wherein the maximum object size stored in any two storage pools is different; and generating index information based on the object size of the object to be processed and the storage location of the target storage pool. In the implementation of the above scheme, since the maximum object size stored in at least two of the multiple storage pools is different, even if any object data in the multiple storage pools is deleted, the idle space after the deletion can be reused by other objects. That is, the idle space can be stored and occupied by target objects smaller than the maximum object size of the storage pool, thus avoiding the impact of re-merging on the target object and improving the timeliness of responding to the target object.

[0012] This application embodiment also provides an object processing apparatus, including: an index information parsing module, used to obtain a data operation request sent by a target device, and parse index information from the data operation request, the index information including: object size and storage location; an object size filtering module, used to filter a target storage pool from multiple storage pools according to the object size, wherein at least two of the multiple storage pools have different maximum object sizes; and a data operation execution module, used to obtain a target object from the target storage pool according to the storage location, and perform corresponding operations on the target object according to the data operation request.

[0013] Optionally, in this embodiment of the application, the index information parsing module includes: a file name parsing submodule, used to parse the file name from the data operation request; and an index information lookup submodule, used to look up the index information corresponding to the file name in the database, wherein the database stores the correspondence between file names and index information.

[0014] Optionally, in this embodiment, the multiple storage pools include: a first storage pool and a second storage pool, wherein the maximum object size stored in the first storage pool is smaller than the maximum object size stored in the second storage pool; the object size filtering module includes: an object size judgment submodule, used to judge whether the object size meets a preset condition, the preset condition including: the object size is greater than the maximum object size stored in the first storage pool and the object size is smaller than the maximum object size stored in the second storage pool; and a target storage pool determination submodule, used to determine the second storage pool as the target storage pool if the object size meets the preset condition, otherwise, determine the first storage pool as the target storage pool.

[0015] Optionally, in this embodiment, the storage pool includes: multiple files, each file including multiple file segments, the size of the file segments being equal to the maximum object size stored in the target storage pool; and a data operation module including: a target file search submodule, used to find the target file containing the target object from the multiple files in the target storage pool based on the storage location; and a target object reading submodule, used to find the target file segment from the multiple file segments in the target file based on the storage location, and read the target object from the target file segment.

[0016] Optionally, in this embodiment, the data operation request is a query request; the data operation module includes: a query response generation submodule, used to generate a query response corresponding to the query request based on index information and the target object, and send the query response corresponding to the query request to the target device.

[0017] Optionally, in this embodiment of the application, the data operation request is a deletion request; the data operation module includes: a target object deletion submodule, used to delete the target object from the target storage pool.

[0018] Optionally, in this embodiment, the object processing apparatus further includes: a storage request receiving module, configured to receive a storage request sent by a target device and parse the object to be processed and its size from the storage request; an object processing storage module, configured to select a target storage pool from multiple storage pools based on the object size and store the object to be processed in the storage location of the target storage pool, wherein the maximum object size stored in any two storage pools is different; and an index information generation module, configured to generate index information based on the object size of the object to be processed and the storage location of the target storage pool.

[0019] This application also provides an electronic device, including a processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the machine-readable instructions, when executed by the processor, perform the method described above.

[0020] This application also provides a computer-readable storage medium storing a computer program that is executed by a processor to perform the methods described above.

[0021] Other features and advantages of embodiments of this application will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing embodiments of this application. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 The diagram shown is a flowchart illustrating the object processing method provided in an embodiment of this application.

[0024] Figure 2 The diagram shown is a structural schematic of the storage engine provided in an embodiment of this application;

[0025] Figure 3 The diagram shown illustrates the bitmaps corresponding to multiple file segments provided in the embodiments of this application.

[0026] Figure 4 The diagram shown is a structural schematic of the object processing apparatus provided in an embodiment of this application;

[0027] Figure 5 The diagram shows a structural schematic of an electronic device provided in an embodiment of this application. Detailed Implementation

[0028] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. The components of the embodiments of this application described and shown in the accompanying drawings can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed embodiments of this application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of the embodiments of this application.

[0029] It is understood that the terms "first" and "second" in the embodiments of this application are used to distinguish similar objects. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that the terms "first" and "second" are not necessarily different.

[0030] Before introducing the object processing method provided in the embodiments of this application, let's first introduce some concepts involved in the embodiments of this application:

[0031] Object storage is a computer data storage architecture that manages data as objects. Unlike other storage architectures, such as file systems which manage data as a file hierarchy, block storage manages data as blocks within sectors and tracks.

[0032] Redis (Remote Dictionary Server) is an open-source, network-enabled, in-memory, distributed, optional persistent key-value store database written in ANSI C.

[0033] It should be noted that the object processing method provided in this application embodiment can be executed by an electronic device. Here, an electronic device refers to a device terminal or server with the function of executing computer programs. Device terminals include, for example, smartphones, personal computers, tablets, personal digital assistants, or mobile internet devices. A server refers to a device that provides computing services through a network. Servers include, for example, x86 servers and non-x86 servers. Non-x86 servers include, for example, mainframes, minicomputers, and UNIX servers.

[0034] The following describes the application scenarios for this object processing method. These scenarios include, but are not limited to, using this method to improve the storage space utilization of object storage devices (OSDs), avoiding the impact of re-merging on the target object, and thus improving the timeliness of responding to the target object. Of course, in specific implementations, this object processing method can also be used to enhance the functionality of object storage devices (OSDs), such as reducing space waste after file deletion and avoiding periodic data re-merging, thereby improving storage space utilization and reuse rates.

[0035] Please see Figure 1 The illustrated flowchart shows an object processing method provided in an embodiment of this application; this embodiment of the application provides an object processing method, including:

[0036] Step S110: Obtain the data operation request sent by the target device, and parse the index information from the data operation request. The index information includes: object size and storage location.

[0037] It is understood that the target device can be a terminal device or a data operation request sent by a server, and the specific type of target device here should not be used as a limitation of the embodiments of this application.

[0038] Step S120: Select a target storage pool from multiple storage pools based on the object size, wherein at least two of the multiple storage pools have different maximum object sizes.

[0039] Please see Figure 2 The diagram illustrates the structure of a storage engine provided in this embodiment of the application. The electronic device described above may include a storage engine, which may include a database and multiple storage pools. The database may be a key-value (KV) database to store index information, and the multiple storage pools are used to store object data corresponding to the index information. It is understood that the electronic device can select a target storage pool from the multiple storage pools in the storage engine based on the object size. At least two storage pools may have different maximum object sizes. Alternatively, the maximum object sizes of any two storage pools may be set to be different, for example: Figure 2 The multiple storage pools include: a 4KB storage pool (for storing objects smaller than 4KB), an 8KB storage pool (for storing objects from 4KB to 8KB), ..., a 1024KB storage pool (for storing objects from 512KB to 1024KB).

[0040] Step S130: Obtain the target object from the target storage pool according to the storage location, and perform corresponding operations on the target object according to the data operation request.

[0041] It is understood that the above-mentioned operations on the target object based on the data operation request include, but are not limited to: query operations, deletion operations, and modification operations, etc. The specific operation process will be described in detail below.

[0042] In the above implementation process, by setting the maximum object size in at least two of the multiple storage pools to be different, since the maximum object size stored in at least two of the multiple storage pools is different, even if any object data in multiple storage pools is deleted, the free space after the object data is deleted can be reused by other objects. That is, the free space can be stored and occupied by target objects smaller than the maximum object size of the storage pool. Therefore, the impact of re-merging on the target object is avoided, thereby improving the timeliness of responding to the target object operation.

[0043] As an optional implementation of step S110 above, when parsing index information from a data operation request, the index information can be searched based on the file name. This implementation may include:

[0044] Step S111: Obtain the data operation request sent by the target device.

[0045] For example, the implementation of step S111 above is as follows: the electronic device receives a data operation request sent by the target device through the Transmission Control Protocol (TCP) or the User Datagram Protocol (UDP). The data operation request here includes, but is not limited to, query requests, deletion requests, and modification requests. These three types of requests will be described in detail below.

[0046] Step S112: Parse the file name from the data operation request.

[0047] For example, in the above-mentioned step S112, the electronic device receives a data operation request sent by the target device via Hypertext Transfer Protocol (HTTP) or Hypertext Transfer Protocol Secure (HTTPS), and then parses the file name from the data operation request.

[0048] Step S113: Search the database for the index information corresponding to the file name. The database stores the correspondence between file names and index information.

[0049] The implementation of step S113 above is as follows: The index information corresponding to the file name is searched in a key-value (KV) database such as Memcached or Redis. This KV database stores the correspondence between file names and index information. Then, the object size and storage location can be parsed from this index information. Specifically, assuming the index information is represented using 16 bytes, the first 8 bytes and the last 8 bytes can be parsed from these 16 bytes. The first 8 bytes represent the object size, and the last 8 bytes represent the object's storage location in the target storage pool. Assuming the specific value of the storage location is S, if the file corresponding to this storage location is divided into N file segments, then the file number where the object is stored is S / N+1, meaning the target object is stored in the S%Nth file segment of the S / N+1th file in the target storage pool.

[0050] As an optional implementation of step S120 above, the multiple storage pools may include: a first storage pool and a second storage pool, wherein the maximum object size stored in the first storage pool is smaller than the maximum object size stored in the second storage pool. In practice, a storage pool is not typically created for every object size; instead, storage pools are usually created systematically, for example... Figure 2 The storage pools are 4KB, 8KB, 16KB, 32KB, 64KB, 128KB, 256KB, 512KB, and 1MB (i.e., 1024KB). The above implementation of filtering the target storage pool from multiple storage pools based on object size may include:

[0051] Step S121: Determine whether the object size meets the preset conditions. The preset conditions include: the object size is greater than the maximum object size stored in the first storage pool and the object size is less than the maximum object size stored in the second storage pool.

[0052] An example implementation of step S121 above is as follows: It is understood that since the maximum object size of each of the multiple storage pools can be set according to specific circumstances, for ease of understanding and explanation, this example uses a 4KB storage pool as an example and an 8KB storage pool as an example. The electronic device can determine whether the object size in the data operation request meets preset conditions. These preset conditions include: the object size is greater than the maximum object size stored in the 4KB storage pool, and the object size is less than the maximum object size stored in the 8KB storage pool.

[0053] Step S122: If the size of the object meets the preset conditions, then the second storage pool is determined as the target storage pool.

[0054] For example, in the implementation of step S122 above: if the object size in the data operation request is 6KB, then the object size meets the preset conditions, that is, the object size is greater than the maximum object size stored in the 4KB storage pool and less than the maximum object size stored in the 8KB storage pool. Then it can be determined that the target object corresponding to the data operation request is stored in the 8KB storage pool, that is, the second storage pool is determined as the target storage pool.

[0055] Step S123: If the size of the object does not meet the preset conditions, then the first storage pool is determined as the target storage pool.

[0056] For example, in the implementation of step S123 above: if the object size in the data operation request is 3KB, then the object size does not meet the preset condition, that is, the object size is smaller than the maximum object size stored in the 4KB storage pool. Then it can be determined that the target object corresponding to the data operation request is stored in the 4KB storage pool, that is, the first storage pool is determined as the target storage pool.

[0057] As an optional implementation method for obtaining the target object in step S130 above, please refer to Figure 2 The aforementioned storage pool may include multiple files, each file comprising multiple file segments. The size of each file segment is equal to the maximum object size stored in the target storage pool; that is, the size of the file segment is the maximum object size that the target storage pool can store. When retrieving a target object from the target storage pool based on its storage location, the target object can be read from the file segments of the files. This implementation may include:

[0058] Step S131: Locate the target file containing the target object from multiple files in the target storage pool based on the storage location.

[0059] Please see Figure 3The illustration shows a schematic diagram of a bitmap corresponding to multiple file segments provided in an embodiment of this application. During the search for a target file segment, a bitmap can be used to help quickly locate it. For example, assuming a file is divided into 1024 file segments, 1024 bits are needed to represent each segment, and the bitmap will occupy an additional 128 bytes of space. When a file segment in the file is used (i.e., the file segment stores an object), the corresponding position in the bitmap can be set to 1. Conversely, when a file segment in the file is released (e.g., the object stored in the file segment is deleted), the corresponding position in the bitmap can be set to 0. When the storage engine of an electronic device needs to allocate a file segment for an object to be processed, it can first search the bitmap for unused file segments. If no unused file segments are found, a new file and its corresponding bitmap (with all bits in the bitmap set to 0) can be created directly, and the newly created file and bitmap can then be used to store the object to be processed.

[0060] For example, the implementation of step S131 above is as follows: If the specific value of the storage location is S, then if the file corresponding to the storage location is divided into N file segments, then the file number of the object storage is S / N+1, that is, the target object is stored in the S%Nth file segment of the S / N+1th file in the target storage pool.

[0061] Understandably, in a storage pool with multiple files, each object can occupy a segment within that file. So, assuming a gigabyte file is divided into multiple segments, when these segments don't store any objects, they are considered file holes, which don't consume disk space. When an object is stored in a file segment, the disk space it occupies is the object's size, not the entire file segment. For example, in a 512KB storage pool, a file segment is 512KB. If an object is 300KB, it will occupy 300KB of disk space, not 512KB. Although the entire 512KB of the file segment is occupied and cannot store other data, the remaining 212KB is a file hole and doesn't consume disk space, so there's no waste. When the object is deleted, the file segment it occupied can be released, freeing up the disk space. The corresponding bit in the bitmap is set to zero, allowing the storage engine in the electronic device to reclaim and reuse the file segment without needing to merge it again or consuming additional storage resources.

[0062] Step S132: Locate the target file segment from multiple file segments in the target file based on the storage location, and read the target object from the target file segment.

[0063] An example implementation of step S132 above is as follows: The target file segment is located from multiple file segments within the target file based on its storage location, and the target object is read from the target file segment. The file segments can be set according to specific circumstances. For example, in a 4KB storage pool (i.e., a storage pool with a 4KB file size), a 1GB file can be divided into multiple file segments, resulting in 1GB / 4KB = 262,144 file segments. This means that a 1GB file can store a maximum of 262,144 objects. Similarly, in a 1MB storage pool (i.e., a storage pool with a 1024KB file size), 1GB / 1024KB = 1024 file segments can be divided, meaning that a 1GB file can store a maximum of 1024 objects.

[0064] In the above implementation, a large file of 1 GB is used to store small objects of different sizes (e.g., small objects of 4KB, 8KB, 16KB, 32KB, 64KB, 128KB, 256KB, 512KB, and 1MB). By storing objects in multiple file segments included in the storage pool, the total number of files in the file system is reduced, and the disk space utilization and read / write performance of the storage engine are improved.

[0065] As a first optional implementation of the corresponding operation in step S130 above, the data operation request is a query request; therefore, the corresponding operation performed on the target object according to the data operation request can be a query operation, and this implementation may include:

[0066] Step S133: Generate a query response corresponding to the query request based on the index information and the target object, and send the query response corresponding to the query request to the target device.

[0067] For example, in the implementation of step S133 above, the electronic device generates a query response corresponding to the query request based on the index information and the target object, and sends the query response corresponding to the query request to the target device via HTTP or HTTPS protocol.

[0068] As a second optional implementation of the corresponding operation in step S130 above, the data operation request is a deletion request; therefore, performing the corresponding operation on the target object according to the data operation request can be a deletion operation, and this implementation may include:

[0069] Step S134: Delete the target object from the target storage pool.

[0070] The implementation of step S134 above includes: In a first implementation, the electronic device deletes the target object from the target storage pool and sends the deletion result to the target device. In a second implementation, the electronic device first backs up the target object, deletes the target object from the target storage pool, and then sends the deletion result to the target device.

[0071] As a third optional implementation of the corresponding operation in step S130 above, the data operation request is a modification request; therefore, performing the corresponding operation on the target object according to the data operation request can be a modification operation, and this implementation may include:

[0072] Step S135: Modify the target object according to the modification request to obtain the modified object.

[0073] Step S136: Determine whether the size of the modified object is greater than the maximum object size stored in the target storage pool.

[0074] The implementation of steps S135 to S136 above is as follows: An executable program compiled or interpreted using a preset programming language modifies the target object according to the modification request, obtains the modified object, and determines whether the size of the modified object is greater than the maximum object size stored in the target storage pool. The programming languages ​​that can be used include, for example, C, C++, Java, BASIC, JavaScript, LISP, Shell, Perl, Ruby, Python, and PHP, etc.

[0075] Step S137: If the size of the modified object is less than or equal to the maximum size of the object stored in the target storage pool, then the modified object will overwrite the target object in the target storage pool.

[0076] Step S138: If the size of the modified object is greater than the maximum size of the object stored in the target storage pool, then select a new storage pool from multiple storage pools based on the size of the modified object, store the modified object in the new storage pool, and then delete the target object in the target storage pool.

[0077] As an optional implementation of the above object processing method, the target object can be stored in the target storage pool before performing corresponding operations on it. This implementation may include:

[0078] Step S141: The electronic device receives the storage request sent by the target device and parses the object to be processed and the object size of the object to be processed from the storage request.

[0079] For example, when an electronic device needs to write the received object to be processed to the storage engine, the electronic device can first receive the storage request sent by the target device via TCP or UDP protocol, and parse the object to be processed and the object size of the object to be processed from the storage request.

[0080] Step S142: The electronic device selects the target storage pool from multiple storage pools based on the object size, and stores the object to be processed in the storage location of the target storage pool. The maximum object size stored in any two storage pools of the multiple storage pools is different.

[0081] For example, in the implementation of step S142 above, the electronic device can select a suitable target storage pool based on the object storage. Specifically, the target storage pool is selected from multiple storage pools based on the object size, and the object to be processed is stored in the storage location of the target storage pool. The maximum object size stored in any two storage pools of the multiple storage pools is different.

[0082] Step S143: The electronic device generates index information based on the object size of the object to be processed and the storage location of the target storage pool.

[0083] For example, the above-mentioned implementation of step S143 is as follows: The index information can be set according to specific circumstances. The index information includes the object size and storage location. For example, assuming that the index information is represented by 16 bytes, the first 8 bytes of the 16 bytes can be used to represent the object size, and the last 8 bytes of the 16 bytes can be used to represent the storage location of the object in the target storage pool.

[0084] Please see Figure 4 The diagram shown is a structural schematic of an object processing apparatus provided in an embodiment of this application; this application provides an object processing apparatus 200, including:

[0085] The index information parsing module 210 is used to obtain the data operation request sent by the target device and parse the index information from the data operation request. The index information includes: object size and storage location.

[0086] The object size filtering module 220 is used to filter the target storage pool from multiple storage pools based on the object size, wherein the maximum object size stored in at least two of the multiple storage pools is different.

[0087] The data operation module 230 is used to retrieve the target object from the target storage pool according to the storage location, and perform corresponding operations on the target object according to the data operation request.

[0088] Optionally, in this embodiment of the application, the index information parsing module includes:

[0089] The file name parsing submodule is used to parse the file name from the data operation request.

[0090] The index information lookup submodule is used to find the index information corresponding to the file name in the database. The database stores the correspondence between file names and index information.

[0091] Optionally, in this embodiment, the multiple storage pools include: a first storage pool and a second storage pool, wherein the maximum object size stored in the first storage pool is smaller than the maximum object size stored in the second storage pool; the object size filtering module includes:

[0092] The object size determination submodule is used to determine whether the object size meets the preset conditions. The preset conditions include: the object size is greater than the maximum object size stored in the first storage pool and the object size is less than the maximum object size stored in the second storage pool.

[0093] The target storage pool determination submodule is used to determine the second storage pool as the target storage pool if the object size meets the preset conditions, otherwise, the first storage pool is determined as the target storage pool.

[0094] Optionally, in this embodiment, the storage pool includes: multiple files, each file including multiple file segments, the size of which is equal to the maximum object size stored in the target storage pool; and a data operation module, including:

[0095] The target file search submodule is used to find the target file containing the target object from multiple files in the target storage pool based on the storage location.

[0096] The target object reading submodule is used to locate the target file segment from multiple file segments in the target file based on the storage location, and read the target object from the target file segment.

[0097] Optionally, in this embodiment, the data operation request is a query request; the data operation module includes:

[0098] The query response generation submodule is used to generate a query response corresponding to the query request based on the index information and the target object, and send the query response corresponding to the query request to the target device.

[0099] Optionally, in this embodiment, the data operation request is a deletion request; the data operation module includes:

[0100] The target object deletion submodule is used to delete target objects from the target storage pool.

[0101] Optionally, in embodiments of this application, the object processing apparatus further includes:

[0102] The storage request receiving module is used to receive storage requests sent by the target device and parse the object to be processed and its size from the storage request.

[0103] The object storage module is used to select the target storage pool from multiple storage pools based on the object size, and store the object to be processed in the storage location of the target storage pool. The maximum object size stored in any two storage pools is different.

[0104] The index information generation module is used to generate index information based on the object size of the object to be processed and the storage location of the target storage pool.

[0105] It should be understood that this device corresponds to the object processing method embodiment described above and is capable of performing the various steps involved in the method embodiment. The specific functions of this device can be found in the description above, and detailed descriptions are omitted here to avoid repetition. The device includes at least one software functional module that can be stored in memory or embedded in the device's operating system (OS) in the form of software or firmware.

[0106] Please see Figure 5 The diagram shows a structural schematic of an electronic device provided in an embodiment of this application. An electronic device 300 provided in this application includes a processor 310 and a memory 320. The memory 320 stores machine-readable instructions executable by the processor 310. When the machine-readable instructions are executed by the processor 310, the method described above is performed.

[0107] This application embodiment also provides a computer-readable storage medium 330, on which a computer program is stored. This computer program is executed by a processor 310 to perform the methods described above. The computer-readable storage medium 330 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0108] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For apparatus embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0109] It should be understood that the disclosed apparatus and methods can also be implemented in other ways, as provided in the embodiments of this application. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the accompanying drawings. For example, two consecutive blocks may actually be executed substantially in parallel, or they may sometimes be executed in reverse order, depending primarily on the functions involved.

[0110] Furthermore, the functional modules of each embodiment in this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part. In addition, in the description of this specification, the reference to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., means that the specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0111] The above description is only an optional implementation of the embodiments of this application, but the protection scope of the embodiments of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the embodiments of this application should be covered within the protection scope of the embodiments of this application.

Claims

1. An object processing method, characterized in that, include: Obtain a data operation request sent by the target device, and parse index information from the data operation request. The index information includes: object size and storage location. A target storage pool is selected from multiple storage pools based on the object size, wherein at least two of the multiple storage pools have different maximum object sizes. The target object is retrieved from the target storage pool according to the storage location, and the corresponding operation is performed on the target object according to the data operation request; The storage pool includes multiple files, each of which includes multiple file segments. The size of each file segment is equal to the size of the largest object stored in the target storage pool. An object occupies one file segment and only occupies disk space equal to the actual size of the object. File segments without stored objects form file holes and do not occupy disk space. When an object in the storage pool is deleted, the file segment it occupies is released.

2. The method according to claim 1, characterized in that, The step of parsing the index information from the data operation request includes: The file name is parsed from the data operation request; The database is used to search for the index information corresponding to the file name, and the database stores the correspondence between the file name and the index information.

3. The method according to claim 1, characterized in that, The plurality of storage pools includes: a first storage pool and a second storage pool, wherein the maximum object size stored in the first storage pool is smaller than the maximum object size stored in the second storage pool; the step of selecting a target storage pool from the plurality of storage pools based on the object size includes: Determine whether the object size meets preset conditions, the preset conditions including: the object size is greater than the maximum object size stored in the first storage pool, and the object size is less than the maximum object size stored in the second storage pool; If so, the second storage pool is determined as the target storage pool; otherwise, the first storage pool is determined as the target storage pool.

4. The method according to claim 1, characterized in that, The step of retrieving the target object from the target storage pool based on the storage location includes: Based on the storage location, the target file containing the target object is located from multiple files in the target storage pool; The target file segment is located from multiple file segments in the target file according to the storage location, and the target object is read from the target file segment.

5. The method according to claim 1, characterized in that, The data operation request is a query request; the step of performing corresponding operations on the target object according to the data operation request includes: Generate a query response corresponding to the query request based on the index information and the target object, and send the query response corresponding to the query request to the target device.

6. The method according to claim 1, characterized in that, The data operation request is a deletion request; the step of performing corresponding operations on the target object according to the data operation request includes: Delete the target object from the target storage pool.

7. The method according to any one of claims 1-6, characterized in that, Also includes: Receive a storage request sent by the target device, and parse the object to be processed and the object size of the object to be processed from the storage request; The target storage pool is selected from the plurality of storage pools based on the object size, and the object to be processed is stored in the storage location of the target storage pool. The maximum object size stored in any two of the plurality of storage pools is different. The index information is generated based on the object size of the object to be processed and the storage location of the target storage pool.

8. An object processing apparatus, characterized in that, include: The index information parsing module is used to obtain data operation requests sent by the target device and parse index information from the data operation requests. The index information includes: object size and storage location. An object size filtering module is used to filter a target storage pool from multiple storage pools based on the object size, wherein the maximum object size stored in at least two of the multiple storage pools is different; The data operation module is used to retrieve the target object from the target storage pool according to the storage location, and perform corresponding operations on the target object according to the data operation request; The storage pool includes multiple files, each of which includes multiple file segments. The size of each file segment is equal to the size of the largest object stored in the target storage pool. An object occupies one file segment and only occupies disk space equal to the actual size of the object. File segments without stored objects form file holes and do not occupy disk space. When an object in the storage pool is deleted, the file segment it occupies is released.

9. An electronic device, characterized in that, include: A processor and a memory, the memory storing machine-readable instructions executable by the processor, which, when executed by the processor, perform the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the method as described in any one of claims 1 to 7.

Images