A data storage, data retrieval method, device and electronic equipment

Abstract

Embodiments of the present application provide a data storage method, a data retrieval method, a data storage device, and an electronic device. The data storage method comprises determining data to be stored; determining a data category of the data to be stored, and continuously arranging the data to be stored belonging to the same data category; and storing the data to be stored in the form of a first data file to a designated storage space in a sequence according to the arrangement order of the data to be stored until the data amount of the data to be stored that has not been stored is less than a designated data amount, wherein the data amount of each first data file is the designated data amount. The data storage method provided by the embodiments of the present application can improve the efficiency of data retrieval.

Description

Technical Field

[0001] This invention relates to the field of data retrieval technology, and in particular to a data storage, data retrieval method, apparatus, and electronic device. Background Technology

[0002] In the era of big data, databases store massive amounts of data, allowing users to retrieve the data they need.

[0003] In related technologies, the common method for data retrieval in a database is to traverse the data stored in the database to determine whether the traversed data meets the data filtering conditions, thereby serving as the target data that matches the data retrieval request.

[0004] However, in the aforementioned related technologies, since databases often store a large amount of data, iterating through each piece of data in the database will consume a lot of time, resulting in low data retrieval efficiency. This is especially true when the data retrieval task is large, where the low data retrieval efficiency is even more pronounced. Summary of the Invention

[0005] The purpose of this invention is to provide a data storage and retrieval method, apparatus, and electronic device to improve the efficiency of data retrieval. The specific technical solution is as follows:

[0006] In a first aspect, embodiments of the present invention provide a data storage method, the method comprising:

[0007] Determine the data to be stored;

[0008] Determine the data category of the data to be stored, and arrange the data to be stored belonging to the same data category consecutively;

[0009] According to the order in which the data to be stored is arranged, the data to be stored is sequentially stored in the form of first data files in the designated storage space until the amount of unstored data to be stored is less than the designated amount of data, wherein the amount of data in each first data file is the designated amount of data.

[0010] Optionally, in one specific implementation, the method further includes:

[0011] Merge the specified files that meet the preset merging conditions stored in the specified storage space to obtain the target file;

[0012] The specified file includes: the first data file, or a first data file in which all the stored data to be stored belongs to the same data category; the preset merging condition includes at least one of the following conditions: the sum of the number of data stored within the preset merging period that reaches a preset value and the amount of the included data to be stored reaches a preset data amount.

[0013] Optionally, in one specific implementation, the step of storing the data to be stored sequentially in the form of a first data file to a designated storage space according to the order of the data to be stored, until the amount of unstored data to be stored is less than the designated amount, includes:

[0014] For each data category, according to the order in which the data to be stored belongs to that data category, the data to be stored belonging to that data category is stored in the designated storage space in the form of the first data file, until the amount of unstored data belonging to that data category is less than the designated data amount.

[0015] Optionally, in one specific implementation, the method further includes:

[0016] If there is unstored data to be stored and the amount of unstored data to be stored is less than the specified amount of data, then the unstored data to be stored is stored in the specified storage space in the form of a second data file.

[0017] Alternatively, in one specific implementation,

[0018] Before storing the data to be stored in the designated storage space in the form of a first data file according to the order of the data to be stored, until the amount of unstored data to be stored is less than the designated amount of data, the method further includes:

[0019] Select data files in the specified storage space whose data volume is less than the specified data volume as the files to be filled;

[0020] Determine the data category of the data in the file to be filled, and use it as the data category for filling;

[0021] The data to be stored, whose data category is the same as the data category to be filled, is stored in the file to be filled until any one of a plurality of preset conditions is met; wherein, the plurality of conditions include a first condition and a second condition, the first condition being: the amount of data in the file to be filled reaches the specified amount of data, and the second condition being: there is no data to be stored in the data to be stored whose data category is the same as the data category to be filled;

[0022] The step of storing the data to be stored sequentially in the form of a first data file to a designated storage space according to the order of the data to be stored, until the amount of unstored data to be stored is less than the designated amount, includes:

[0023] For the remaining data to be stored that has not been stored in the file to be filled, the remaining data to be stored is stored in the specified storage space in the form of the first data file according to the order of the remaining data to be stored, until the amount of the unstored data to be stored is less than the specified amount of data.

[0024] Optionally, in one specific implementation, the designated storage space is an SSD with a PCIe interface.

[0025] Secondly, embodiments of the present invention provide a data retrieval method, the method comprising:

[0026] Obtain a data retrieval request and determine the target data category of the data requested in the data retrieval request;

[0027] In the target storage space, a target file is determined that stores data belonging to the target data category, wherein the data in the target storage space is stored according to any of the data storage methods provided in the first aspect above;

[0028] Based on the search criteria indicated by the data retrieval request, target data matching the search request is searched from the target file.

[0029] Thirdly, embodiments of the present invention provide a data storage device, the device comprising:

[0030] The data determination module is used to determine the data to be stored.

[0031] A data sorting module is used to determine the data category of the data to be stored and to sort the data to be stored belonging to the same data category consecutively.

[0032] The first data storage module is used to sequentially store the data to be stored in the form of first data files to a designated storage space according to the arrangement order of the data to be stored, until the amount of unstored data to be stored is less than the designated amount of data, wherein the amount of data in each first data file is the designated amount of data.

[0033] Optionally, in one specific implementation, the apparatus further includes:

[0034] The file merging module is used to merge specified files stored in the specified storage space that meet preset merging conditions to obtain a target file; wherein, the specified file includes: the first data file, or, a first data file in which all the data to be stored belongs to the same data category; the preset merging conditions include at least one of the following conditions: the sum of the number of data stored within the preset merging period that reaches a preset value and the amount of the data to be stored included reaches a preset data amount.

[0035] Optionally, in one specific implementation, the first data storage module is specifically used for:

[0036] For each data category, according to the order of the data to be stored belonging to that data category, each specified amount of data belonging to that data category is stored in the specified storage space in the form of the first data file, until the amount of unstored data belonging to that data category is less than the specified amount of data.

[0037] Optionally, in one specific implementation, the apparatus further includes:

[0038] The second data storage module is used to store the unstored data to be stored in the specified storage space in the form of a second data file when there is unstored data to be stored and the amount of the unstored data to be stored is less than the specified amount of data.

[0039] Optionally, in one specific implementation, the apparatus further includes:

[0040] The file determination module is used to determine data files in the specified storage space whose data volume is less than the specified data volume, and to select them as files to be filled.

[0041] The category determination module is used to determine the data category of the data in the file to be filled, and use it as the data category to be filled.

[0042] The file filling module is used to store data whose data category is the same as the filling data category in the data to be stored into the file to be filled, until any one of a plurality of preset conditions is met; wherein, the plurality of conditions include a first condition and a second condition, the first condition being: the data volume in the file to be filled reaches the specified data volume, and the second condition being: there is no data to be stored whose data category is the same as the filling data category in the data to be stored.

[0043] The first data storage module is specifically used for:

[0044] For the remaining data to be stored that has not been stored in the file to be filled, the remaining data to be stored is stored in the specified storage space in the form of the first data file according to the order of the remaining data to be stored, until the amount of the unstored data to be stored is less than the specified amount of data.

[0045] Optionally, in one specific implementation, the designated storage space is an SSD with a PCIe interface.

[0046] Fourthly, embodiments of the present invention provide a data retrieval device, the device comprising:

[0047] The request acquisition module is used to acquire data retrieval requests and determine the target data category of the data requested in the data retrieval request;

[0048] The target retrieval module is used to determine, in the target storage space, a target file containing data belonging to the target data category, wherein the data in the target storage space is stored according to any of the data storage methods provided in the first aspect above;

[0049] The data retrieval module is used to search for target data that matches the retrieval request from the target file based on the retrieval conditions indicated by the data retrieval request.

[0050] Fifthly, embodiments of the present invention provide an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0051] Memory, used to store computer programs;

[0052] A processor, when executing a program stored in memory, implements any of the data storage method steps provided in the first aspect and / or the data retrieval method steps provided in the second aspect.

[0053] In a sixth aspect, embodiments of the present invention provide a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements any of the data storage method steps provided in the first aspect and / or the data retrieval method steps provided in the second aspect.

[0054] In a seventh aspect, embodiments of the present invention also provide a computer program product containing instructions that, when run on a computer, cause the computer to perform any of the data storage method steps provided in the first aspect and / or the data retrieval method steps provided in the second aspect.

[0055] Beneficial effects of the embodiments of the present invention:

[0056] As can be seen from the above, by applying the solution provided in the embodiments of the present invention, during the data storage process, the data to be stored can be determined first, then the data category of the data to be stored can be determined, and the data to be stored belonging to the same data category can be arranged continuously. Thus, after the arrangement is completed, multiple data to be stored of the same data type are continuous, and each of these multiple data to be stored can be adjacent to at least one other data to be stored with the same data category. Furthermore, the data to be stored can be stored sequentially in the form of first data files to a designated storage space according to the order of the data to be stored, until the amount of unstored data is less than a designated amount. Here, the amount of data in each first data file is the designated amount, and the data to be stored with the same data category in each first data file is continuous.

[0057] Accordingly, when performing data retrieval, after obtaining a data retrieval request, the target data category of the data requested by the retrieval request can be determined first; then, in the specified storage space, the target file containing data belonging to the target data category can be determined; and then, based on the retrieval conditions indicated by the data retrieval request, the target data matching the retrieval request can be found in the target file.

[0058] Based on this, after data storage is performed using the data storage method provided in this embodiment of the invention, during the data retrieval process, since the data belonging to the target data category of the target data to be retrieved is centrally stored in a portion of the files within the specified storage space, the number of files storing data belonging to the target data category in the specified storage space can be reduced. Furthermore, data in files storing data belonging to the target data category can be compared without comparing all data in all files, thus reducing the amount of data compared and the number of I / O (Input / Output) operations during the data retrieval process. Moreover, when a file storing data belonging to the target data category contains data of more than one data category, since the data belonging to the same data category in the file is continuous, when retrieving data from that file, the data belonging to the target data category in the file can be compared sequentially according to the order of arrangement of the data belonging to the target data category, without comparing all data in the target file, thereby further reducing the amount of data compared during the data retrieval process. Furthermore, since data comparison and I / O both consume time during the data retrieval process, the above data storage method in this embodiment of the invention can reduce the amount of data retrieved and the number of I / O operations during the data retrieval process. Therefore, the time required for data comparison can be shortened and the efficiency of data retrieval can be improved during the data retrieval process. Attached Figure Description

[0059] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0060] Figure 1 This is a schematic diagram of a first process for a data storage method provided in an embodiment of the present invention;

[0061] Figure 2 This is a schematic diagram of a second process for a data storage method provided in an embodiment of the present invention;

[0062] Figure 3(a) is a schematic diagram of the various data to be stored obtained in a specific embodiment;

[0063] Figure 3(b) is a schematic diagram of determining the data category for each piece of data to be stored in Figure 3(a);

[0064] Figure 3(c) is a schematic diagram of the continuous arrangement of the data to be stored for each data category in Figure 3(b);

[0065] Figure 3(d) is a schematic diagram showing how the data to be stored in each data category after being arranged consecutively in Figure 3(c) are stored in the form of data files.

[0066] Figure 3(e) is a schematic diagram of merging the various data files in Figure 3(d);

[0067] Figure 4 This is a schematic diagram of a third process for a data storage method provided in an embodiment of the present invention;

[0068] Figure 5 This is a schematic diagram of the fourth process of the data storage method provided in the embodiments of the present invention;

[0069] Figure 6 A schematic flowchart of a data retrieval method provided in an embodiment of the present invention;

[0070] Figure 7 This is a schematic diagram of the structure of a data storage device provided in an embodiment of the present invention;

[0071] Figure 8 This is a schematic diagram of the structure of a data retrieval device provided in an embodiment of the present invention;

[0072] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0073] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of the present invention.

[0074] In related technologies, data retrieval in a database typically involves iterating through the stored data to determine if the retrieved data meets the filtering criteria, thus identifying the target data that matches the retrieval request. However, in these technologies, since databases often store large amounts of data, iterating through each piece of data is time-consuming, resulting in low data retrieval efficiency.

[0075] To address the aforementioned problems, embodiments of the present invention provide a data storage method.

[0076] It should be noted that this method can be applied to various data storage scenarios that serve data retrieval needs, and the stored objects can be various types of data such as images, text, video, or audio. That is, the data to be stored in this embodiment of the invention can be various types of data such as image data, text data, audio data, or video data. For example, storing surveillance videos to facilitate subsequent viewing of various events occurring in the monitored area, or storing road traffic recordings to facilitate subsequent viewing of the causes of traffic accidents, etc. Therefore, this embodiment of the invention does not specifically limit the application scenarios or the data types of the data to be stored.

[0077] Furthermore, the executing entity of this invention can be any type of electronic device capable of performing data storage operations, such as a mobile phone, a laptop computer, or a desktop computer; wherein, the electronic device can be an independent electronic device or a device cluster composed of multiple electronic devices. This invention does not specifically limit the specific type of electronic device, and will be referred to as an electronic device.

[0078] One data storage method provided in this embodiment of the invention may include the following steps:

[0079] Determine the data to be stored;

[0080] Determine the data category of the data to be stored, and arrange the data to be stored belonging to the same data category consecutively;

[0081] According to the order in which the data to be stored is arranged, the data to be stored is sequentially stored in the form of first data files in the designated storage space until the amount of unstored data to be stored is less than the designated amount of data, wherein the amount of data in each first data file is the designated amount of data.

[0082] As can be seen from the above, by applying the solution provided in the embodiments of the present invention, during the data storage process, the data to be stored can be determined first, then the data category of the data to be stored can be determined, and the data to be stored belonging to the same data category can be arranged continuously. Thus, after the arrangement is completed, multiple data to be stored of the same data type are continuous, and each of these multiple data to be stored can be adjacent to at least one other data to be stored with the same data category. Furthermore, the data to be stored can be stored sequentially in the form of first data files to a designated storage space according to the order of the data to be stored, until the amount of unstored data is less than a designated amount. Here, the amount of data in each first data file is the designated amount, and the data to be stored with the same data category in each first data file is continuous.

[0083] Based on this, after data storage is performed using the data storage method provided in this embodiment of the invention, during the data retrieval process, since the data belonging to the target data category of the target data to be retrieved is centrally stored in a portion of the data files within the specified storage space, the number of files storing data belonging to the target data category in the specified storage space can be reduced. Furthermore, data in files storing data belonging to the target data category can be compared without comparing all data in all files, thus reducing the amount of data compared and the number of I / O operations during the data retrieval process. Moreover, when a file storing data belonging to the target data category contains data of more than one data category, since the data belonging to the same data category in the file is continuous, when retrieving data from that file, the data belonging to the target data category in the file can be compared sequentially according to the arrangement order of the data belonging to the target data category, without comparing all data in the target file, thereby further reducing the amount of data compared during the data retrieval process. Furthermore, since data comparison and I / O consume time during the data retrieval process, the above-described data storage method in this embodiment of the invention can reduce the amount of data retrieved and the number of I / O operations during the data retrieval process. Therefore, the time required for data comparison can be shortened and the efficiency of data retrieval can be improved during the data retrieval process.

[0084] The following is a detailed description of a data storage method provided by an embodiment of the present invention, with reference to the accompanying drawings.

[0085] Figure 1 This is a schematic diagram of a first type of data storage method provided in an embodiment of the present invention, such as... Figure 1 As shown, the method may include the following steps S101-S103.

[0086] S101: Determine the data to be stored.

[0087] When storing data, it is necessary to first determine the data to be stored. There are various ways to determine the data to be stored, and this embodiment of the invention does not impose specific limitations on these methods.

[0088] For example, data acquired in real time can be identified as data to be stored.

[0089] For example, previously acquired but not stored data can be identified as data to be stored. For instance, if data previously acquired by an electronic device was not stored due to device malfunction or other reasons, this data can be identified as data to be stored after the device returns to normal operation.

[0090] For example, during the previous data storage process, some data may not have been stored. After real-time data acquisition, both the real-time acquired data and the aforementioned unstored data can be identified as data to be stored.

[0091] The data source for the data to be stored can be various electronic devices that can generate or transmit data.

[0092] Optionally, the data to be stored can be data received by the electronic device from other devices. For example, the electronic device can receive video data sent by a monitoring device and then identify the received video data as the data to be stored.

[0093] Optionally, the electronic device may also identify its own generated data as the data to be stored. For example, when the electronic device is a monitoring device with data storage capabilities, it may identify its own collected video data as the data to be stored.

[0094] S102: Determine the data category of the data to be stored, and arrange the data to be stored belonging to the same data category consecutively.

[0095] Electronic devices can pre-obtain the classification method of the data to be stored. Therefore, after determining the data to be stored, they can first classify the data according to the above classification method to determine the data category of each data to be stored.

[0096] The above classification method can determine the data categories from various dimensions according to the needs of actual applications.

[0097] For example, the data to be stored can be divided into video data, image data, text data, audio data, etc.

[0098] For example, data from the same source can be classified into the same data category, while data from different sources can be classified into different data categories.

[0099] Furthermore, after determining the data category of the data to be stored, the data belonging to the same data category can be arranged consecutively, thus grouping the data of the same category together. In this way, after the arrangement is complete, multiple data items of the same data type are consecutive, and each of these multiple data items can be adjacent to at least one other data item of the same data category.

[0100] For example, the electronic device can retrieve each piece of data to be stored in real time, and the order in which the data to be stored is retrieved is as follows:

[0101] Image a, video b, audio c, audio d, image e, and audio f;

[0102] Furthermore, after determining the data categories of the data to be stored according to the classification methods of images, videos, and audio, the data to be stored belonging to the same data category can be arranged consecutively, resulting in the following arrangement order: image a, image e, video b, audio c, audio d, and audio f.

[0103] Typically, electronic devices can cache data to be stored in a designated storage space in their own memory.

[0104] Optionally, when determining the data category of each piece of data to be stored, data of the same category can be cached in the same data shard.

[0105] If the total number of data categories to be stored is not greater than the number of data fragments in the memory of the electronic device, then the data to be stored belonging to each data category can be cached in a separate data fragment.

[0106] Correspondingly, if the total number of data categories to be stored is greater than the number of data shards in the memory of the electronic device, then each data shard can cache multiple data categories to be stored. Thus, for each data shard, the data to be stored belonging to the same data category in the cached data of that data shard can be arranged consecutively to achieve the arrangement of data to be stored with the same data category together in sequence.

[0107] S103: According to the order of the data to be stored, store the data to be stored in the specified storage space in the form of the first data file until the amount of unstored data to be stored is less than the specified amount of data.

[0108] The data size of each first data file is a specified amount.

[0109] After arranging the data to be stored in the same data category consecutively, in order to store the data of the same data category in a more concentrated and orderly manner, the data to be stored in the same data category should be stored in one file as much as possible.

[0110] Therefore, the data to be stored can be stored in the specified storage space in the form of first data files according to the order of the data to be stored, until the amount of unstored data is less than the specified amount of data, and the amount of data in each first data file is the specified amount of data.

[0111] In other words, starting with the first piece of data to be stored that has not yet been stored, each piece of data to be stored can be identified as belonging to the same first data file in turn, until the amount of data to be stored that has been identified reaches a specified amount. Then, the specified amount of data to be stored that belongs to the same first data file can be stored in the specified storage space in the form of a first data file. This process is repeated in turn, storing each first data file in the specified storage space in turn, until the amount of data to be stored that has not yet been stored is less than the specified amount.

[0112] Since the amount of each piece of data to be stored may not be an integer multiple of the specified amount of data, not all the data to be stored may be stored in the specified storage space in the form of a first data file with the specified amount of data. Therefore, during the storage process, there may be data to be stored that is smaller than the specified amount of data that is not stored in the specified storage space.

[0113] For example, there are data A1, A2 and A3 belonging to data category A, data B1 and B2 belonging to data type B, and data C1 and C2 belonging to data category C. After the data to be stored belonging to the same data category are arranged consecutively, the order of the data to be stored is: A1, A2, A3, B1, B2, C1 and C2, and the amount of data in A1, A2, A3, B1, B2, C1 and C2 is the same.

[0114] If the data volume of the three data items to be stored is taken as the specified data volume, then according to the above arrangement, A1, A2, and A3 are read in sequence first, and then A1, A2, and A3 are stored in the specified storage space as a first data file. Then, B1, B2, and C1 are read in sequence, and B1, B2, and C1 are stored in the specified storage space as a first data file. After that, since only one data item C2 has not been stored, and since the data volume of C2 is less than the specified data volume, the data storage process ends here, and there is one unstored data item C2.

[0115] Since the data in each data category of the data to be stored may not be an integer multiple of the specified data amount, when applying the above-described storage procedure, some of the first data files stored in the specified storage space may contain data to be stored in at least two data categories. Therefore, in order to place data of the same data category in the same first data file, so that each first data file can contain data of only one data category, after arranging the data to be stored belonging to the same data category consecutively, the data to be stored in each data category can also be stored separately.

[0116] Optionally, in one specific implementation, step S103 may include the following step one:

[0117] Step 1: For each data category, according to the order of the data to be stored belonging to that data category, store the data to be stored belonging to that data category in the form of a first data file to the specified storage space, until the amount of unstored data belonging to that data category is less than the specified amount of data.

[0118] In this specific implementation, after the data to be stored belonging to the same data category are arranged consecutively, for each data category, the data to be stored belonging to that data category can be stored in the form of a first data file in the specified storage space according to the arrangement order of the data to be stored belonging to that data category, until the amount of unstored data belonging to that data category is less than the specified amount of data.

[0119] The storage process for the data to be stored for each data category is similar to the data storage process in step S103 above, and will not be described again here.

[0120] Similarly, since the amount of data to be stored in each data category may not be an integer multiple of the specified data amount, not all the data to be stored in that data category may be stored in the specified storage space in the form of a first data file with the specified data amount. Therefore, during the storage process, there may be data to be stored in that data category with a data amount less than the specified data amount that is not stored in the specified storage space.

[0121] For example, there are data A1, A2 and A3 belonging to data category A, data B1 and B2 belonging to data type B, and data C1 and C2 belonging to data category C. After the data to be stored belonging to the same data category are arranged consecutively, the order of the data to be stored is: A1, A2, A3, B1, B2, C1 and C2, and the amount of data in A1, A2, A3, B1, B2, C1 and C2 is the same.

[0122] If the data volume of the three data items to be stored is taken as the specified data volume, then according to the above arrangement order, A1, A2, and A3 are read in sequence, and A1, A2, and A3 are stored in the specified storage space as a first data file. After that, since the sum of the data volume of B1 and B2 is less than the specified data volume, and the sum of the data volume of C1 and C2 is also less than the specified data volume, the data storage process ends here, and there are unstored data items B1, B2, C1, and C2 to be stored.

[0123] The specified amount of data can be determined by those skilled in the art based on the actual situation, and the embodiments of the present invention do not impose specific limitations on this.

[0124] The aforementioned specified storage space can be various types of hardware capable of storing data, such as various hard drives, floppy disks, optical discs, flash memory, USB flash drives, MMC cards (Multimedia Cards), MS cards (Memory Sticks), etc. Furthermore, the aforementioned specified storage space can also be cloud storage space; however, this embodiment of the invention does not specifically limit its use. Here, USB is an abbreviation for Universal Serial Bus.

[0125] Optionally, in one specific implementation, the specified storage space can be an SSD (Solid State Disk) with a PCIe (Peripheral Component Interconnect Express) interface.

[0126] In this specific implementation, compared to other hard drives, SSDs with PCIe interfaces can have better I / O performance. Therefore, using SSDs with PCIe interfaces as designated storage space can further improve the data read and write efficiency during the data retrieval process, thereby further improving the data retrieval efficiency.

[0127] As mentioned earlier, when the amount of data belonging to a certain data category exceeds the specified amount, multiple first data files containing that data category can exist in the specified storage space. In other words, for each data category, multiple first data files containing that category of data may exist in the specified storage space. Therefore, to further reduce I / O operations during retrieval, the first data files can be merged to concentrate data of the same data type into a smaller number of files.

[0128] Alternatively, in one specific implementation, such as Figure 2As shown, the data storage method provided in this embodiment of the invention may further include the following step S201.

[0129] S201: Merge specified files that meet preset merging conditions stored in the specified storage space to obtain the target file.

[0130] The specified file includes: a first data file, or a first data file in which all the data to be stored belongs to the same data category; the preset merging conditions include at least one of the following conditions: the sum of the number of data stored within the preset merging period that reaches a preset value and the amount of the data to be stored included reaches a preset data amount.

[0131] In this specific implementation, during the process of storing the data to be stored in the specified storage space in the form of first data files according to the order of the data to be stored, the resulting first data files are stored sequentially in the specified storage space. Thus, the specified files that meet the preset merging conditions stored in the specified storage space can be merged to obtain the target file.

[0132] In the merging process, for each pair of first data files with adjacent storage times, when merging the two first data files with adjacent storage times, the last data to be stored in the first data file with the earlier storage time is adjacent to the first data to be stored in the first data file with the later storage time, and the last data to be stored in the first data file with the earlier storage time is located before the first data to be stored in the first data file with the later storage time.

[0133] In this way, since the data to be stored of the same data type in each first data file is arranged consecutively, and similarly, for each two first data files with adjacent storage times, when not stored, the last data to be stored in the first data file with the earlier storage time is adjacent to the first data to be stored in the first data file with the later storage time, and the last data to be stored in the first data file with the earlier storage time is located before the first data to be stored in the first data file with the later storage time, then after merging the two first data files with adjacent storage times, the order of the data to be stored in the merged file belonging to the two first data files with adjacent storage times is the same as the order of the data to be stored in the two first data files with adjacent storage times when not stored.

[0134] Based on this, since multiple data to be stored of the same data type are consecutive after being arranged in a continuous manner, and each of these multiple data to be stored can be adjacent to at least one other data to be stored of the same data type, after merging the specified files to obtain the target file, the multiple data to be stored of the same data type in the obtained target file are consecutive, and each of these multiple data to be stored can be adjacent to at least one other data to be stored of the same data type.

[0135] Optionally, the specified file can be any of the stored first data files, thereby merging the first data files stored in the specified storage space within a preset merging period to obtain the target file.

[0136] For example, if the preset merging period is 1 day, the first data file stored in the specified storage space between 0:00 and 24:00 on February 22, 2022 can be merged to obtain a target file. This target file contains all the data contained in the first data file stored in the specified storage space between 0:00 and 24:00 on February 22, 2022.

[0137] Optionally, the specified file can be any of the stored first data files. Thus, when the number of first data files stored in the specified storage space reaches a preset value, the preset value of first data files can be merged to obtain the target file.

[0138] For example, when the number of first data files stored in the specified storage space reaches 10, the 10 first data files are merged to obtain a target file. The target file contains all the data contained in the 10 first data files.

[0139] Optionally, the specified file can be any of the stored first data files. Thus, when the sum of the data to be stored in the first data files stored in the specified storage space reaches a preset data amount, the first data files whose sum of the data to be stored reaches a preset data amount can be merged to obtain the target file.

[0140] Optionally, the specified file can be a first data file in which all the data to be stored belongs to the same data category. Thus, the first data files in which all the data to be stored belongs to the same data category and are stored in the specified storage space within a preset merging period can be merged to obtain the target file.

[0141] Optionally, the specified file can be a first data file in which all the stored data belongs to the same data category. Thus, when the number of first data files in the specified storage space in which all the stored data belongs to the same data category reaches a preset value, the preset value of first data files in the specified storage space in which all the stored data belongs to the same data category are merged to obtain the target file.

[0142] For example, if the number of first data files containing all the data to be stored in a specified storage space that belong to category A reaches 10, these 10 first data files can be merged to obtain a target file that only stores category A data. This target file contains all the category A data contained in the 10 first data files.

[0143] Optionally, the specified file can be a first data file in which all the data to be stored belongs to the same data category. Thus, when the sum of the data of the first data file in which all the data to be stored belongs to the same data category and is stored in the specified storage space reaches a preset data amount, the first data file in which all the data to be stored belongs to the same data category and is stored in the specified storage space can be merged to obtain the target file.

[0144] To facilitate understanding of the data storage method provided in the above embodiments of the present invention, it can be... Figures 3(a)-3(e) The schematic diagram shown illustrates an example of a data storage method provided in the above-described embodiment of the present invention.

[0145] Step 1: As shown in Figure 3(a), each data to be stored, A1-A4, B1-B4, C1-C4, D1-D4, E1-E4, and F1-F4, can be obtained;

[0146] Step 2: As shown in Figure 3(b), the data to be stored is classified according to data categories A, B, C and D, and the data to be stored in each data category is cached into different data fragments.

[0147] Step 3: As shown in Figure 3(c), the data categories cached in each data slice are arranged consecutively;

[0148] Step 4: As shown in Figure 3(d), for each data segment, the consecutively arranged data to be stored in the data segment is stored in the specified storage space as a first data file; where n = 4.

[0149] Step 5: As shown in Figure 3(e), the three consecutive first data files stored in the specified storage space are merged to obtain the target file; among them, the target files marked A+B+C include: A1-A4, B1-B4, and C1-C4, and the target files marked D+E+F include: D1-D4, E1-E4, and F1-F4.

[0150] Based on this, after data storage is performed using the data storage method provided in this embodiment of the invention, during the data retrieval process, since the data belonging to the target data category of the target data to be retrieved is centrally stored in a portion of the data files within the specified storage space, the number of files storing data belonging to the target data category in the specified storage space can be reduced. Furthermore, data in files storing data belonging to the target data category can be compared without comparing all data in all files, thus reducing the amount of data compared and the number of I / O operations during the data retrieval process. Moreover, when a file storing data belonging to the target data category contains data of more than one data category, since the data belonging to the same data category in the file is continuous, when retrieving data from that file, the data belonging to the target data category in the file can be compared sequentially according to the arrangement order of the data belonging to the target data category, without comparing all data in the target file, thereby further reducing the amount of data compared during the data retrieval process. Furthermore, since data comparison and I / O consume time during the data retrieval process, the above-described data storage method in this embodiment of the invention can reduce the amount of data retrieved and the number of I / O operations during the data retrieval process. Therefore, the time required for data comparison can be shortened and the efficiency of data retrieval can be improved during the data retrieval process.

[0151] As described in the various specific implementation methods above, after data storage, some data to be stored may not be stored because the specified data volume has not been reached, and thus continue to be stored in memory. In order to save the memory of electronic devices, this data can also be stored to free up memory space.

[0152] Alternatively, in one specific implementation, such as Figure 4 As shown, the data storage method provided in this embodiment of the invention may further include the following step S401.

[0153] S401: If there is unstored data to be stored and the amount of unstored data to be stored is less than the specified amount of data, then the unstored data to be stored is stored in the specified storage space in the form of a second data file.

[0154] For existing unstored data that is less than the specified data size, the unstored data can be stored in the specified storage space as a second data file. Thus, after each data storage, there will be no unstored data remaining in the memory space, thereby saving memory in electronic devices.

[0155] In addition, if there is unstored data to be stored, then when retrieving data in the specified storage space, it will be impossible to compare the unstored data to be stored, which may result in the inability to find the target data being retrieved. Therefore, when storing data each time, unstored data to be stored that is less than the specified data size is stored in the specified storage space in the form of a second data file, which can also improve the accuracy and real-time performance of the retrieval results.

[0156] In such Figure 4 Based on the specific implementation shown, when there is unstored data to be stored and the amount of unstored data is less than the specified data size, the unstored data smaller than the specified data size is stored in the specified storage space as a second data file. While this can save memory space and improve the accuracy and real-time performance of data retrieval results, for the same amount of data, if some data files are smaller than the specified data size, the number of data files may increase. This results in more data files needing to be searched during data retrieval, thus making the improvement in data retrieval efficiency less significant. Therefore, to maximize data retrieval efficiency while saving memory and improving the accuracy and real-time performance of retrieval results, a better approach is to first fill the specified storage space with data files whose data size is less than the specified data size using the data to be stored.

[0157] Alternatively, in one specific implementation, such as Figure 5 As shown, the data storage method provided in this embodiment of the invention may further include the following steps S501-S503.

[0158] S501: Identify data files in the specified storage space whose data volume is less than the specified data volume, and use them as files to be filled.

[0159] After arranging the data to be stored in the same data category consecutively, it can be determined first whether there are data files in the specified storage space with a data size smaller than the specified data size.

[0160] The file to be filled can be any data file in the specified storage space whose data volume is less than the specified data volume. For example, a second data file whose last stored data volume was less than the specified data volume; or a data file that has been filled before but has not yet been filled to the specified data volume. This embodiment of the invention does not impose specific limitations on this.

[0161] If there is no data file with a data size smaller than the specified data size in the specified storage space, then step S103 above can be executed directly. The data to be stored is stored in the specified storage space in the form of a first data file according to the order of the data to be stored, until the data size of the unstored data to be stored is smaller than the specified data size.

[0162] Correspondingly, if they exist, data files with a data size smaller than the specified data size in the specified storage space can be identified as files to be filled.

[0163] The number of files to be filled can be one or more.

[0164] S502: Determine the data category of the data in the file to be filled, and use it as the data category for filling.

[0165] Once the file to be filled is identified, the data category of the data in the file can be determined and used as the data category for filling.

[0166] If the file to be filled contains data of a certain data category, then the data category of the data contained in the file to be filled can be determined as the data category to be filled.

[0167] If the file to be filled includes data from multiple data categories, the data category to be filled can be randomly determined from among these multiple data categories, or the data category with the largest storage address in the file to be filled can be determined as the data category to be filled.

[0168] S503: Store the data to be stored in the data to be stored that has the same data category as the data to be filled into the file to be filled, until any one of the preset conditions is met.

[0169] Among them, the above conditions include a first condition and a second condition. The first condition is that the amount of data in the file to be filled reaches the specified amount of data. The second condition is that there is no data to be stored in the data to be stored that has the same data category as the data to be filled.

[0170] After determining the data categories to be filled, the data of the same category as the data to be filled can be stored in the file to be filled. When using the stored data of the same category as the data to be filled to populate the file, the following situations may occur:

[0171] The first scenario: If the amount of data in the data to be stored that shares the same data category as the data to be filled is not less than the difference between the amount of data in the file to be filled and the specified amount of data, then when using the data in the data to be stored that shares the same data category as the data to be filled to populate the file, the amount of data in the file to be filled can be increased to the specified amount of data. In this case, the data in the data to be stored that shares the same data category as the data to be filled can be stored in the file to be filled until the amount of data in the file to be filled reaches the specified amount of data.

[0172] The second scenario: The amount of data in the data to be stored that shares the same data category as the data to be filled is less than the difference between the amount of data in the file to be filled and the specified amount of data. In this case, when using the data in the data to be stored that shares the same data category as the data to be filled to populate the file, the amount of data in the file to be filled cannot be increased to the specified amount of data. In this situation, the data in the data to be stored that shares the same data category as the data to be filled can be stored in the file to be filled until there is no more data in the data to be stored that shares the same data category as the data to be filled.

[0173] Accordingly, step S103 above, which involves storing the data to be stored in the specified storage space in the form of a first data file according to the order of the data to be stored, until the amount of unstored data to be stored is less than the specified amount of data, may include the following step S504:

[0174] S504: For the remaining data to be stored that has not been stored in the file to be filled, store each specified amount of data to be stored in the specified storage space in the form of a first data file according to the order of the remaining data to be stored, until the amount of data to be stored that has not been stored is less than the specified amount of data.

[0175] After filling the file to be filled with the data to be stored, the remaining data to be stored that has not been stored in the file to be filled can be stored in the specified storage space in the form of a first data file according to the order of the remaining data to be stored, until the amount of unstored data to be stored is less than the specified amount of data.

[0176] The specific implementation of step S504 is similar to that of step S103, and will not be repeated here.

[0177] The following section will continue to describe a data retrieval method provided by an embodiment of the present invention.

[0178] This data retrieval method can be applied to various data retrieval scenarios, and the retrieved data can be image data, text data, audio data, or video data, etc.

[0179] Furthermore, the entity executing this data retrieval method can be any electronic device capable of performing data storage operations, such as a mobile phone, laptop, or desktop computer. This electronic device can be a standalone device or a cluster of multiple electronic devices; this embodiment of the invention does not specifically limit its use, and will be referred to as the retrieval device. The aforementioned retrieval device and the entity executing the aforementioned data storage method can be the same device or different devices.

[0180] Figure 6 This is a schematic flowchart of a data retrieval method provided in an embodiment of the present invention.

[0181] S601: Obtain a data retrieval request and determine the target data category of the data requested in the data retrieval request.

[0182] During the data retrieval process, you can first obtain the data retrieval request and determine the data category of the data requested in the data retrieval request as the target data category.

[0183] The aforementioned data retrieval request can be a data retrieval request sent by a user based on various clients, a retrieval request sent by various devices, or a self-retrieval request from the server itself. This embodiment of the invention does not specifically limit the scope of the request.

[0184] In addition, the above-mentioned data retrieval request can take many forms. For example, the data retrieval request can be text information, voice information, and image information, etc. The embodiments of the present invention do not make specific limitations.

[0185] S602: In the target storage space, determine the target file that stores data belonging to the target data category.

[0186] The data in the target storage space is stored according to any of the above data storage methods.

[0187] After determining the target data category of the data requested in the data retrieval request, the target file containing data belonging to the target data category can be found in the target storage space based on the target data category.

[0188] S603: Based on the search criteria indicated by the data retrieval request, search for target data in the target file that matches the retrieval request.

[0189] Once the target file containing data belonging to the target data category has been identified, the target data matching the search request can be found in the target file based on the search criteria indicated by the data retrieval request.

[0190] Based on this, after data storage is performed using the data storage method provided in this embodiment of the invention, during the data retrieval process, since the data belonging to the target data category of the target data to be retrieved is centrally stored in a portion of the data files within the specified storage space, the number of files storing data belonging to the target data category in the specified storage space can be reduced. Furthermore, data in files storing data belonging to the target data category can be compared without comparing all data in all files, thus reducing the amount of data compared and the number of I / O operations during the data retrieval process. Moreover, when a file storing data belonging to the target data category contains data of more than one data category, since the data belonging to the same data category in the file is continuous, when retrieving data from that file, the data belonging to the target data category in the file can be compared sequentially according to the arrangement order of the data belonging to the target data category, without comparing all data in the target file, thereby further reducing the amount of data compared during the data retrieval process. Furthermore, since data comparison and I / O consume time during the data retrieval process, the above-described data storage method in this embodiment of the invention can reduce the amount of data retrieved and the number of I / O operations during the data retrieval process. Therefore, the time required for data comparison can be shortened and the efficiency of data retrieval can be improved during the data retrieval process.

[0191] Corresponding to the data storage method provided in the above embodiments of the present invention, the present invention also provides a data storage device.

[0192] Figure 7 This is a schematic diagram of the structure of a data storage device provided in an embodiment of the present invention, such as... Figure 7 As shown, the data storage device may include the following modules:

[0193] Data determination module 701 is used to determine the data to be stored;

[0194] The data arrangement module 702 is used to determine the data category of the data to be stored and to arrange the data to be stored belonging to the same data category consecutively.

[0195] The first data storage module 703 is used to store the data to be stored in the form of a first data file to a specified storage space in the order of the data to be stored, until the amount of unstored data to be stored is less than the specified amount of data.

[0196] The data volume of each of the first data files is the specified data volume.

[0197] Based on this, after data storage is performed using the data storage method provided in this embodiment of the invention, during the data retrieval process, since the data belonging to the target data category of the target data to be retrieved is centrally stored in a portion of the data files within the specified storage space, the number of files storing data belonging to the target data category in the specified storage space can be reduced. Furthermore, data in files storing data belonging to the target data category can be compared without comparing all data in all files, thus reducing the amount of data compared and the number of I / O operations during the data retrieval process. Moreover, when a file storing data belonging to the target data category contains data of more than one data category, since the data belonging to the same data category in the file is continuous, when retrieving data from that file, the data belonging to the target data category in the file can be compared sequentially according to the arrangement order of the data belonging to the target data category, without comparing all data in the target file, thereby further reducing the amount of data compared during the data retrieval process. Furthermore, since data comparison and I / O consume time during the data retrieval process, the above-described data storage method in this embodiment of the invention can reduce the amount of data retrieved and the number of I / O operations during the data retrieval process. Therefore, the time required for data comparison can be shortened and the efficiency of data retrieval can be improved during the data retrieval process.

[0198] Optionally, in one specific implementation, the above-mentioned data storage device may further include:

[0199] The file merging module is used to merge specified files stored in the specified storage space that meet preset merging conditions to obtain a target file; wherein, the specified file includes: the first data file, or, a first data file in which all the data to be stored belongs to the same data category; the preset merging conditions include at least one of the following conditions: the sum of the number of data stored within the preset merging period that reaches a preset value and the amount of the data to be stored included reaches a preset data amount.

[0200] Optionally, in one specific implementation, the first data storage module is specifically used for:

[0201] For each data category, according to the order of the data to be stored belonging to that data category, each specified amount of data belonging to that data category is stored in the specified storage space in the form of the first data file, until the amount of unstored data belonging to that data category is less than the specified amount of data.

[0202] Optionally, in one specific implementation, the above-mentioned data storage device may further include:

[0203] The second data storage module is used to store the unstored data to be stored in the specified storage space in the form of a second data file when there is unstored data to be stored and the amount of the unstored data to be stored is less than the specified amount of data.

[0204] Optionally, in one specific implementation, the above-mentioned data storage device may further include:

[0205] The file determination module is used to determine data files in the specified storage space whose data volume is less than the specified data volume, and to select them as files to be filled.

[0206] The category determination module is used to determine the data category of the data in the file to be filled, and use it as the data category to be filled.

[0207] The file filling module is used to store data whose data category is the same as the filling data category in the data to be stored into the file to be filled, until any one of a plurality of preset conditions is met; wherein, the plurality of conditions include a first condition and a second condition, the first condition being: the data volume in the file to be filled reaches the specified data volume, and the second condition being: there is no data to be stored whose data category is the same as the filling data category in the data to be stored.

[0208] The first data storage module is specifically used for:

[0209] For the remaining data to be stored that has not been stored in the file to be filled, the remaining data to be stored is stored in the specified storage space in the form of the first data file according to the order of the remaining data to be stored, until the amount of the unstored data to be stored is less than the specified amount of data.

[0210] Optionally, in one specific implementation, the designated storage space in the above data storage device is an SSD with a PCIe interface.

[0211] Corresponding to the data retrieval method provided in the above embodiments of the present invention, the present invention also provides a data retrieval device.

[0212] Figure 8 This is a schematic diagram of the structure of a data retrieval device provided in an embodiment of the present invention, as shown below. Figure 8 As shown, the data retrieval device may include the following modules:

[0213] The request acquisition module 801 is used to acquire a data retrieval request and determine the target data category of the data requested by the retrieval request;

[0214] The target retrieval module 802 is used to determine, in the target storage space, a target file containing data belonging to the target data category, wherein the data in the target storage space is stored according to any data storage method provided in the embodiments of the present invention;

[0215] The data retrieval module 803 is used to search for target data that matches the retrieval request from the target file based on the retrieval conditions indicated by the data retrieval request.

[0216] Based on this, after data storage is performed using the data storage method provided in this embodiment of the invention, during the data retrieval process, since the data belonging to the target data category of the target data to be retrieved is centrally stored in a portion of the data files within the specified storage space, the number of files storing data belonging to the target data category in the specified storage space can be reduced. Furthermore, data in files storing data belonging to the target data category can be compared without comparing all data in all files, thus reducing the amount of data compared and the number of I / O operations during the data retrieval process. Moreover, when a file storing data belonging to the target data category contains data of more than one data category, since the data belonging to the same data category in the file is continuous, when retrieving data from that file, the data belonging to the target data category in the file can be compared sequentially according to the arrangement order of the data belonging to the target data category, without comparing all data in the target file, thereby further reducing the amount of data compared during the data retrieval process. Furthermore, since data comparison and I / O consume time during the data retrieval process, the above-described data storage method in this embodiment of the invention can reduce the amount of data retrieved and the number of I / O operations during the data retrieval process. Therefore, the time required for data comparison can be shortened and the efficiency of data retrieval can be improved during the data retrieval process.

[0217] This invention also provides an electronic device, such as... Figure 9 As shown, it includes a processor 901, a communication interface 902, a memory 903, and a communication bus 904, wherein the processor 901, the communication interface 902, and the memory 903 communicate with each other through the communication bus 904.

[0218] Memory 903 is used to store computer programs;

[0219] The processor 901, when executing the program stored in the memory 903, implements the steps of any data storage method and / or data retrieval method provided in the embodiments of the present invention.

[0220] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not indicate that there is only one bus or one type of bus.

[0221] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0222] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0223] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0224] In another embodiment of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored therein, and when the computer program is executed by a processor, it implements the steps of any of the above-described data storage methods and / or data retrieval methods.

[0225] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the data storage methods and / or data retrieval methods described in the above embodiments.

[0226] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0227] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0228] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device embodiments, electronic device embodiments, computer-readable storage medium embodiments, and computer program product embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0229] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A data storage method, characterized in that, The method includes: Determine the data to be stored; Determine the data category of the data to be stored, and arrange the data to be stored belonging to the same data category consecutively; A data file in a specified storage space with a data volume smaller than a specified data volume is identified as the file to be filled; the file to be filled is: the second data file whose data volume in the previous storage was smaller than the specified data volume. Determine the data category of the data in the file to be filled, and use it as the data category for filling; The data to be stored, whose data category is the same as the data category to be filled, is stored in the file to be filled until any one of a plurality of preset conditions is met; wherein, the plurality of conditions include a first condition and a second condition, the first condition being: the amount of data in the file to be filled reaches the specified amount of data, and the second condition being: there is no data to be stored in the data to be stored whose data category is the same as the data category to be filled; For the remaining data to be stored that has not been stored in the file to be filled, the remaining data to be stored is stored in the designated storage space in the form of a first data file according to the order of the remaining data to be stored, until the amount of unstored data to be stored is less than the designated amount of data, wherein the amount of data in each first data file is the designated amount of data; If there is unstored data to be stored and the amount of unstored data to be stored is less than the specified amount of data, then the unstored data to be stored is stored in the specified storage space in the form of a second data file.

2. The method according to claim 1, characterized in that, The method further includes: Merge the specified files that meet the preset merging conditions stored in the specified storage space to obtain the target file; The specified file includes: the first data file, or a first data file in which all the stored data to be stored belongs to the same data category; the preset merging condition includes at least one of the following conditions: the sum of the number of data stored within the preset merging period that reaches a preset value and the amount of the included data to be stored reaches a preset data amount.

3. The method according to claim 1, characterized in that, The specified storage space is an SSD with a PCIe interface.

4. A data retrieval method, characterized in that, The method includes: Obtain a data retrieval request and determine the target data category of the data requested in the data retrieval request; In the target storage space, a target file is determined that stores data belonging to the target data category, wherein the data in the target storage space is stored in accordance with the method described in any one of claims 1-3; Based on the search criteria indicated by the data retrieval request, target data matching the search request is searched from the target file.

5. A data storage device, characterized in that, The device includes: The data determination module is used to determine the data to be stored. A data sorting module is used to determine the data category of the data to be stored and to sort the data to be stored belonging to the same data category consecutively. The file determination module is used to determine data files in a specified storage space whose data volume is less than a specified data volume, and to use them as files to be filled; the files to be filled are: the second data file whose data volume in the last storage was less than the specified data volume; The category determination module is used to determine the data category of the data in the file to be filled, and use it as the data category to be filled. The file filling module is used to store data whose data category is the same as the filling data category in the data to be stored into the file to be filled, until any one of a plurality of preset conditions is met; wherein, the plurality of conditions include a first condition and a second condition, the first condition being: the data volume in the file to be filled reaches the specified data volume, and the second condition being: there is no data to be stored whose data category is the same as the filling data category in the data to be stored; The first data storage module is used to store the remaining data to be stored in the file to be filled in the form of a first data file in a specified storage space according to the order of the remaining data to be stored, until the amount of the unstored data to be stored is less than the specified amount of data, wherein the amount of data in each first data file is the specified amount of data; The second data storage module is used to store the unstored data to be stored in the specified storage space in the form of a second data file when there is unstored data to be stored and the amount of the unstored data to be stored is less than the specified amount of data.

6. The apparatus according to claim 5, characterized in that, The device further includes: The file merging module is used to merge specified files stored in the specified storage space that meet preset merging conditions to obtain a target file; wherein, the specified file includes: the first data file, or, a first data file in which all the data to be stored belongs to the same data category; the preset merging conditions include at least one of the following conditions: the sum of the number of data stored within the preset merging period that reaches a preset value and the amount of the data to be stored included reaches a preset data amount; And / or, The specified storage space is an SSD with a PCIe interface.

7. A data retrieval device, characterized in that, The device includes: The request acquisition module is used to acquire data retrieval requests and determine the target data category of the data requested in the data retrieval request; A target retrieval module is used to determine, in a target storage space, a target file containing data belonging to the target data category, wherein the data in the target storage space is stored according to the method described in any one of claims 1-3; The data retrieval module is used to search for target data that matches the retrieval request from the target file based on the retrieval conditions indicated by the data retrieval request.

8. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method steps of any one of claims 1-3 and / or the method steps of claim 4.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 1-3 and / or the steps of the method described in claim 4.

Images