Attribute classification-based data storage method and related device

Through the data storage method based on attribute classification, the problem of low query efficiency of relational databases under large-scale data is solved. By establishing index relationships and memory disk mapping relationships, efficient data positioning and storage are achieved, improving user experience and query accuracy.

CN118916359APending Publication Date: 2024-11-08BEIJING SOHU NEW MEDIA INFORMATION TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411091758.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-09
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

When facing large-scale data, the query efficiency of relational databases has significantly decreased, and they cannot directly locate data that meets the query conditions, which affects the timeliness of user experience and business decisions.

Method used

Using a data storage method based on attribute classification, classify the stored data by obtaining the target attributes, establish an index relationship between the target attribute value and the corresponding sub-data, locate the data set in the mapping relationship between memory and disk locations according to the index relationship, and optimize the storage and Query process.

Benefits of technology

It significantly improves query efficiency, narrows the query scope, ensures the accuracy of query results, and maintains a low query response time as the data volume continues to increase, improving user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118916359A_ABST
    Figure CN118916359A_ABST
Patent Text Reader

Abstract

The invention discloses a data storage method based on attribute classification and a related device, and the method comprises the steps: obtaining to-be-stored data and target attributes from a target process, then classifying the to-be-stored data according to the target attributes to obtain a plurality of data sets, and sub-data in each data set has the same target attribute value. According to the mapping relation between the memory positions and the disk positions, the target disk position corresponding to the target memory position is determined, and the disk position is used for indicating the storage position of each piece of sub-data in the data set in the file. And finally, if the target disk position has the target data file, storing the plurality of data sets in corresponding storage positions according to the target disk position to obtain a corresponding relationship between the target attribute value and each corresponding sub-data storage position. According to the method, the stored data is classified and stored according to the target attribute, and the data query is performed according to the index relationship between the target attribute value and the corresponding sub-data, so that the query efficiency is remarkably improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data file storage method, device, system and medium based on attribute classification. Background Art

[0002] With the rapid development of the Internet, the amount of data that needs to be stored on various network platforms has increased exponentially. For example, social networks, e-commerce, online games and other platforms generate massive amounts of user behavior data every day, including comments, likes, browsing history, transaction records, etc.

[0003] Most of this data is stored in relational databases, such as MySQL and Oracle. These databases are suitable for processing structured data and ensuring data consistency with their mature ACID features and rich query languages. Relational databases store data in the form of tables. Each table consists of rows and columns. Each row represents a record and each column represents a field.

[0004] When the amount of data in a single table of a relational database is too large, more rows and columns need to be scanned when querying data. In traditional storage methods, it is impossible to directly locate data that meets the query conditions, and the query efficiency is significantly reduced. Summary of the invention

[0005] In view of this, the present application provides a data storage method and related devices based on attribute classification to solve the problem that the query efficiency of the relational database is greatly reduced when facing query requests with large data volumes, thereby improving the query efficiency.

[0006] In a first aspect, the present application provides a data storage method based on attribute classification, the method comprising:

[0007] Acquire data to be stored and a target attribute from a target process, wherein the data to be stored includes a plurality of sub-data, each of which has a plurality of attributes, and the target process indicates a target memory location where the data to be stored is stored; the target attribute is one of the plurality of attributes, and the target attribute is determined based on a query condition;

[0008] Classifying the data to be stored according to the target attribute to obtain multiple data sets, wherein the sub-data in each data set has the same target attribute value;

[0009] Determine the target disk position corresponding to the target memory position according to the mapping relationship between the memory position and the disk position, wherein the disk position is used to indicate the storage position of each sub-data in the data set in the file;

[0010] If the target disk location has a target data file, multiple data sets are stored in corresponding storage locations in the target data file according to the target disk location to obtain a first index relationship, which is a correspondence between the target attribute value and the storage location of each sub-data in the corresponding data set in the target data file.

[0011] In a possible implementation, storing the plurality of data sets in corresponding storage locations in the target data file according to the target disk locations to obtain a first index relationship includes:

[0012] storing the plurality of data sets in storage locations corresponding to the temporary file according to the target disk locations, and obtaining a second index relationship, wherein the second index relationship is a correspondence relationship between the target attribute value and the storage location of the corresponding data set in the temporary file;

[0013] The temporary file is merged with the historical data file at the target disk location to obtain the target data file and the first index relationship.

[0014] In a possible implementation, determining the target disk position corresponding to the target memory position according to the mapping relationship between the memory position and the disk position includes:

[0015] A mapping relationship between a memory location and a disk location is established according to a memory mapping file technology, and a target disk location corresponding to the target memory location is determined.

[0016] In a possible implementation, merging the temporary file with the historical data file at the target disk location to obtain the target data file and the first index relationship includes:

[0017] During the daily low-peak period of network traffic, the temporary file is merged with the historical data file at the target disk location to obtain the target data file and the first index relationship.

[0018] In a possible implementation, the method further includes:

[0019] The data set is stored in the memory to obtain a third index relationship;

[0020] If a query request for the target attribute value is obtained, obtaining a data set corresponding to the target attribute value from the memory according to the third index relationship;

[0021] If the data set corresponding to the target attribute value fails to be obtained from the memory, the data set corresponding to the target attribute value is obtained from the disk location according to the first index relationship.

[0022] In a possible implementation, obtaining the data to be stored from the target process includes:

[0023] Obtain the data to be stored from the target process through an interface, middleware or application.

[0024] A second aspect of the present application provides a data storage device based on attribute classification, the device comprising: an acquisition unit, a classification unit, a positioning unit and a storage unit;

[0025] The acquisition unit is used to acquire the data to be stored and the target attribute from the target process, the data to be stored includes a plurality of sub-data, each sub-data has a plurality of attributes, the target process indicates the target memory location where the data to be stored is stored; the target attribute is one of the plurality of attributes, and the target attribute is determined based on the query condition;

[0026] The classification unit is used to classify the data to be stored according to the target attribute to obtain multiple data sets, and the sub-data in each data set has the same target attribute value;

[0027] The positioning unit is used to determine the target disk position corresponding to the target memory position according to the mapping relationship between the memory position and the disk position, and the disk position is used to indicate the storage position of each sub-data in the data set in the file;

[0028] The storage unit is used to store multiple data sets in corresponding storage positions in the target data file according to the target disk position if there is a target data file, so as to obtain a first index relationship, wherein the first index relationship is a correspondence between the target attribute value and the storage position of each sub-data in the corresponding data set in the target data file.

[0029] A third aspect of the present application provides a computer device, characterized in that the device includes a processor and a memory:

[0030] The memory is used to store program code and transmit the program code to the processor;

[0031] The processor is configured to execute the method described in the first aspect above according to instructions in the program code.

[0032] A fourth aspect of the present application provides a computer-readable storage medium, characterized in that the computer-readable storage medium is used to store a computer program, and the computer program is used to execute the method described in the first aspect above.

[0033] A fifth aspect of the present application provides a computer program product, characterized in that it includes a computer program or instructions; when the computer program or instructions are executed by a processor, the method described in the first aspect above is executed.

[0034] It can be seen from the above technical solutions that the present application has the following beneficial effects:

[0035] By classifying the stored data according to the target attribute, storing the classified data set on disk, and establishing an index relationship between the target attribute value and the corresponding sub-data, when the user queries the target attribute value, the data set that meets the query conditions can be quickly located based on the index relationship, greatly narrowing the query scope, thereby significantly improving the query efficiency, quickly responding to the user's query request, and improving the user experience. Based on the index relationship, all the sub-data of the data set obtained by the user's query have the corresponding target attribute value, ensuring the accuracy of the query result. According to the mapping relationship between the memory location and the disk location, the target disk location corresponding to the target memory location is determined, and the data set is written to the disk. In the process of storing data, the process operates the memory. Based on the characteristics of the continuous arrangement of the target memory location, sequential storage can be achieved, which improves the storage speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A method flow chart of a data storage method based on attribute classification provided in an embodiment of the present application;

[0037] Figure 2 A schematic diagram of the structure of a data storage device based on attribute classification provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be construed as being limited to the embodiments described herein. Instead, these embodiments are provided to provide a more thorough and complete understanding of the present application. It should be understood that the drawings and embodiments of the present application are only for exemplary purposes and are not intended to limit the scope of protection of the present application.

[0039] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0040] In today's digital age, with the rapid development of platforms such as social media, e-commerce, and online games, the amount of data generated by users has shown an explosive growth trend. For example, on an active social platform, hundreds of millions of user interaction events occur every day, including likes, comments, and sharing. These behavioral data are not only large in volume but also diverse in variety, and are usually stored in relational databases. A relational database is a database management system based on a relational model. A relational database organizes data in a tabular form, storing data in a two-dimensional table in the form of rows and columns. Each table consists of a set of records (rows) and attributes (columns). Each row of a table represents a data entity, and each column represents an attribute. Tables are associated with each other through keys (such as primary keys and foreign keys) to achieve data relations. The core features of relational databases include ACID (atomicity, consistency, isolation, and durability) transaction support, which ensures data reliability and consistency. Relational databases use structured query language to define, operate, and query data, and provide a rich set of query languages ​​and operation functions, allowing users to retrieve and operate data through complex query statements. Common relational database systems include MySQL, Oracle, Microsoft SQL Server, and PostgreSQL.

[0041] Network platforms that use relational databases as data storage methods need to be able to efficiently store and query key data such as user click-through rates, operational behaviors, etc. However, as the amount of data grows, the traditional storage methods based on relational databases gradually expose their shortcomings. When the amount of data reaches a certain scale, the large amount of data in a single table of a relational database causes the relational database to scan a large number of rows and columns when performing queries, and it is impossible to directly locate data that meets specific query conditions, which leads to a significant decrease in query efficiency, directly affecting user experience and the timeliness of business decisions.

[0042] Therefore, how to maintain efficient query efficiency when facing large-scale data has become a key issue that needs to be solved urgently.

[0043] In order to solve the above technical problems, it is necessary to find a more efficient data storage solution. The present application discloses a data storage method based on attribute classification. By classifying the stored data according to the target attribute and storing the classified data set on the disk, an index relationship between the target attribute value and the corresponding sub-data is established. When a user queries a specific target attribute value, the system can quickly locate the data set containing the attribute value based on the index relationship, thereby greatly narrowing the query scope and significantly improving the query efficiency. This means that even if the amount of data continues to increase, the query response time can still be maintained at a low level, improving the user experience.

[0044] Exemplary Methods

[0045] See also Figure 1 , Figure 1 A method flow chart of a data storage method based on attribute classification provided in an embodiment of the present application, the method specifically comprises the following steps:

[0046] Step 101: Obtain data to be stored and target attributes from a target process, where the data to be stored includes multiple sub-data, each sub-data has multiple attributes, and the target process indicates a target memory location for storing the data to be stored.

[0047] The target process refers to the application or service that generates the data to be stored, such as a web server that generates log data after processing user requests.

[0048] Data to be stored refers to data that needs to be saved, such as user behavior data, log files, or transaction records.

[0049] Sub-data refers to a single data item in the data to be stored, such as a record of user click behavior.

[0050] Attributes refer to the characteristics or dimensions that describe sub-data, such as user ID, event occurrence, timestamp, etc.

[0051] The target attribute is a specific attribute selected from multiple attributes of the data to be stored. The selection of the attribute is based on the query requirements, that is, it is the attribute that will be used to filter the data when querying.

[0052] The target memory location refers to a piece of memory space corresponding to the specific address of the file on the disk where the data to be stored is to be stored, that is, the location of the data to be stored in the memory. The address of this memory space and the specific address on the disk are mapped through memory mapping technology. Each sub-data has a corresponding target memory location, which corresponds to the location where the sub-data is stored on the disk.

[0053] The network platform needs to record various user behavior data, such as likes, comments, shares, etc. User behavior data needs to be recorded in real time and eventually saved in a persistent storage system for subsequent analysis and use. The target process of the network platform generates data to be stored including user behavior data. The target process indicates the target memory location of the data to be stored. Because the target memory location has a mapping relationship with the location of the storage disk, the target process actually indicates the specific location of the disk where the data to be stored is to be stored. The data to be stored may be one or more sub-data, each of which contains multiple attributes and is acquired by the data storage classification system provided in the embodiment of the present application (i.e., a system using the method provided in the embodiment of the present application).

[0054] For example, user A posts a comment in a social media application. This comment behavior generates a sub-data, which contains multiple attributes, such as user ID, comment content, and comment timestamp. The target process will pass this sub-data as part of the data to be stored to the data storage classification system.

[0055] In a possible implementation, the method of obtaining the data to be stored from the target process provided in the embodiment of the present application may specifically include:

[0056] Obtain the data to be stored from the target process through an interface, middleware or application.

[0057] Acquisition through an interface refers to receiving data through an application programming interface (API), which is often used in microservice architectures, where different services exchange data through preset APIs.

[0058] Acquisition through middleware means receiving data through middleware such as message queues and cache systems. This method is suitable for asynchronous processing scenarios and can effectively decouple producers and consumers.

[0059] Direct application delivery refers to the delivery of data directly through internal program calls, which is common in single applications or tightly coupled services. The method provided in the embodiment of the present application can support multiple data delivery methods such as interfaces, middleware, and applications, so as to flexibly respond to data storage in different scenarios, such as interfaces in scenarios, thereby enhancing the adaptability and flexibility of the attribute data storage classification system.

[0060] Step 102: Classify the data to be stored according to the target attribute to obtain multiple data sets, and the sub-data in each data set has the same target attribute value.

[0061] Each sub-data is classified according to its target attribute value, that is, each sub-data is classified into the corresponding data set according to its target attribute value. Each data set contains all sub-data with the same target attribute value. For example, if the target attribute is "user ID", then all sub-data with the same user ID will be classified into the same data set. Various algorithms can be used to implement the classification operation of the stored data, such as simple loop traversal, hash table, dictionary tree, etc. After the classification of the stored data is completed, multiple data sets will be obtained, each of which contains sub-data with the same target attribute value. The data set will be used for subsequent data storage and index construction processes.

[0062] For example, after step 101, 5 sub-data are obtained from the target process:

[0063] Record 1: {User ID: 1001, Action type: "click", Timestamp: "2024-07-0118:00:00"}

[0064] Record 2: {User ID: 1002, Action type: "search", Timestamp: "2024-07-0118:05:00"}

[0065] Record 3: {User ID: 1001, Action type: "purchase", Timestamp: "2024-07-0118:10:00"}

[0066] Record 4: {User ID: 1001, Action type: "click", Timestamp: "2024-07-0118:15:00"}

[0067] Select user ID as the target attribute and classify the data into different data sets based on user ID:

[0068] The data set with user ID 1001: {record 1, record 3, record 4}

[0069] Data set with user ID 1002: {Record 2}

[0070] After the data to be stored is classified, the sub-data in each data set has the same target attribute value, which means that when building an index, you only need to create an index item containing the target attribute value for each data set to determine the sub-data that meets the target attribute value.

[0071] Step 103: Determine the target disk position corresponding to the target memory position according to the mapping relationship between the memory position and the disk position. The disk position is used to indicate the storage position of each sub-data in the data set in the file.

[0072] The target memory position of each sub-data corresponds to the target disk position one by one, and the target disk position can be determined by the target memory position indicated by the target process and the mapping relationship between the memory position and the disk position.

[0073] In a possible implementation, the method of determining the target disk position corresponding to the target memory position according to the mapping relationship between the memory position and the disk position provided in the embodiment of the present application may specifically include:

[0074] A mapping relationship between a memory location and a disk location is established according to a memory mapping file technology, and a target disk location corresponding to the target memory location is determined.

[0075] Memory-mapped file technology is a mechanism provided by the operating system that can map a file or other object into the address space of a process. When a file is mapped by memory file mapping technology, the program can directly access the contents of the file just like accessing memory, without explicitly using system calls to read and write files. This mechanism can significantly increase the speed of file access because it reduces context switching between user mode and kernel mode and takes advantage of the page cache mechanism of modern operating systems.

[0076] For example, there are 4 sub-data:

[0077] Data example:

[0078] Record 1: {User ID: 1001, Action type: "click", Timestamp: "2024-07-0118:00:00"}

[0079] Record 2: {User ID: 1002, Action type: "search", Timestamp: "2024-07-0118:05:00"}

[0080] Record 3: {User ID: 1001, Action type: "purchase", Timestamp: "2024-07-0118:10:00"}

[0081] Record 4: {User ID: 1001, Action type: "click", Timestamp: "2024-07-0118:15:00"}

[0082] Assume that the data block in the memory is located at address 0x10000. According to the memory-mapped file technology, this memory location corresponds to the location 0x00000 in the disk file data.log.

[0083] For record 1 (user ID is 1001), assuming the memory location is 0x10000, according to the memory mapping file technology, the target disk location corresponding to the memory location 0x10000 is determined to be the location 0x00000 in the file data.log. For record 3 (user ID is 1001), assuming the memory location is 0x20000, according to the memory mapping file technology, the target disk location corresponding to the memory location 0x20000 is determined to be the location 0x00100 in the file data.log. For record 4 (user ID is 1001), assuming the memory location is 0x30000, according to the memory mapping file technology, the target disk location corresponding to the memory location 0x30000 is determined to be the location 0x00200 in the file data.log. For record 2 (user ID is 1002), assuming the memory location is 0x40000, according to the memory mapping file technology, the target disk location corresponding to the memory location 0x40000 is determined to be the location 0x00300 in the file data.log.

[0084] Specifically, the memory file mapping technology can use file mapping (mmap, memory-mapped file).

[0085] Through the memory mapping file technology, the storage location on the disk corresponding to the data in the memory can be quickly determined, which provides the necessary conditions for building an index, because the index relationship records the corresponding relationship between the target attribute value and the storage location of each sub-data in the corresponding data set in the data file. At the same time, most modern operating systems today support memory mapping file technology, which makes the data storage method that implements this mapping relationship deployable on multiple system platforms.

[0086] Step 104: If the target disk location has a target data file, multiple data sets are stored in corresponding storage locations in the target data file according to the target disk location to obtain a first index relationship, which is a correspondence between the target attribute value and the storage location of each sub-data in the corresponding data set in the target data file.

[0087] After storing the sub-data in the multiple data sets in the corresponding storage locations in the data file, a first index relationship is established. In the first index relationship, there is a corresponding relationship between the target attribute value and the storage location of each sub-data of the corresponding data set in the data file. When the user queries the target attribute value, all the sub-data in the disk that match the target attribute value can be obtained according to the index relationship. The existence of the first index relationship makes the query operation more efficient. Even if the amount of data continues to increase, the query response time can still be maintained at a low level.

[0088] For example, if all records with user ID 1001 are queried, the system only needs to access the first index relationship containing user ID 1001 to obtain the storage location of the sub-data with user ID 1001 in the disk data file without scanning the entire data file.

[0089] In a possible implementation, the embodiment of the present application provides storing the plurality of data sets in corresponding storage locations in the target data file according to the target disk location to obtain a first index relationship, which may specifically include:

[0090] storing the plurality of data sets in storage locations corresponding to the temporary file according to the target disk locations, and obtaining a second index relationship, wherein the second index relationship is a correspondence relationship between the target attribute value and the storage location of the corresponding data set in the temporary file;

[0091] The temporary file is merged with the historical data file at the target disk location to obtain the target data file and the first index relationship.

[0092] Temporary files and data files are stored in the disk. Before the merge operation is performed, each data set is first stored in a temporary file on the disk to obtain a second index relationship, which is the correspondence between the target attribute value and the storage location of the corresponding data set in the temporary file. By merging the temporary file with the historical data file, a new data file and the first index relationship are obtained to reflect the index relationship in the new data file.

[0093] Data is frequently generated in the network platform. Using temporary files to store data can reduce the frequent modification of existing data files and improve concurrent processing capabilities. The index relationship records the correspondence between the target attribute value and the storage location of each sub-data, so that the required data can be quickly located during query, and the data set with a specific attribute value can be quickly located, which significantly improves query efficiency. Merging temporary files with data files to obtain new data files and the first index relationship can reduce the occupation of system resources and avoid affecting normal business operations. At the same time, merging files can optimize the use of disk space and reduce fragmentation problems.

[0094] In a possible implementation, the embodiment of the present application provides merging the temporary file with the historical data file at the target disk location to obtain the target data file and the first index relationship, which may specifically include:

[0095] During the daily low-peak period of network traffic, the temporary file is merged with the historical data file at the target disk location to obtain the target data file and the first index relationship.

[0096] During the daily low-peak period of network traffic, such as 4 a.m., scheduled merging operations are performed to minimize the impact of data processing on system resources, especially to reduce the occupancy of network bandwidth, so as not to affect user experience and the operation of other key services.

[0097] Performing data processing tasks during peak network traffic hours may cause network congestion, affecting the speed at which users access websites or use applications. Choosing to perform merge operations during low network traffic hours can help reduce competition for network resources and ensure that network bandwidth is mainly used for user activities. Low traffic hours usually mean that the system load is low, and performing data processing tasks at this time can better utilize server resources and improve overall system performance.

[0098] In a possible implementation, the method provided in this embodiment further includes:

[0099] The data set is stored in the memory to obtain a third index relationship;

[0100] If a query request for the target attribute value is obtained, obtaining a data set corresponding to the target attribute value from the memory according to the third index relationship;

[0101] If the data set corresponding to the target attribute value fails to be obtained from the memory, the data set corresponding to the target attribute value is obtained from the disk location according to the first index relationship.

[0102] The purpose of the above operation is to store the data set in the memory for the temporary query task of the data to be stored before establishing the first index relationship, and obtain the third index relationship. The third index relationship is the correspondence between the position of the sub-data of the data set in the memory and the corresponding target attribute value, and directly query the data set in the memory according to the third index relationship. After the index relationship of the corresponding data set is established, the data can be queried according to the index relationship in order to reduce the memory usage, improve the memory utilization, and delete the data set in the memory.

[0103] The embodiment of the present application classifies the stored data by target attributes, stores the classified data set to the disk, and establishes an index relationship between the target attribute value and the corresponding sub-data. When the user queries the target attribute value, the data set that meets the query conditions can be quickly located according to the index relationship, which greatly narrows the query scope, thereby significantly improving the query efficiency, quickly responding to the user's query request, and improving the user experience. Based on the index relationship, all the sub-data of the data set obtained by the user query have the corresponding target attribute value, which ensures the accuracy of the query result. According to the mapping relationship between the memory location and the disk location, the target disk location corresponding to the target memory location is determined, and the data set is written to the disk. In the process of storing data, the process operates the memory. Based on the characteristics of the continuous arrangement of the target memory location, sequential storage can be achieved, which improves the storage speed.

[0104] Exemplary Devices

[0105] See also Figure 2 , Figure 2 A structural diagram of a data storage device based on attribute classification provided in an embodiment of the present application, the device comprising: an acquisition unit 201, a classification unit 202, a positioning unit 203 and a storage unit 204;

[0106] The acquisition unit 201 is used to acquire the data to be stored and the target attribute from the target process, the data to be stored includes multiple sub-data, each sub-data has multiple attributes, the target process indicates the target memory location where the data to be stored is stored; the target attribute is one of the multiple attributes, and the target attribute is determined based on the query condition;

[0107] A classification unit 202, configured to classify the data to be stored according to the target attribute to obtain multiple data sets, wherein the sub-data in each data set has the same target attribute value;

[0108] A positioning unit 203 is used to determine a target disk position corresponding to a target memory position according to a mapping relationship between a memory position and a disk position, wherein the disk position is used to indicate a storage position of each sub-data in a data set in a file;

[0109] Storage unit 204 is used to store multiple data sets in corresponding storage positions in the target data file according to the target disk position if there is a target data file, and obtain a first index relationship. The first index relationship is the correspondence between the target attribute value and the storage position of each sub-data in the corresponding data set in the target data file.

[0110] In a possible implementation, the storage unit is specifically configured to:

[0111] According to the target disk position, multiple data sets are stored in the storage position corresponding to the temporary file to obtain a second index relationship, where the second index relationship is a correspondence relationship between the target attribute value and the storage position of the corresponding data set in the temporary file;

[0112] The temporary file is merged with the historical data file at the target disk location to obtain the target data file and the first index relationship.

[0113] In a possible implementation manner, the positioning unit is specifically configured to:

[0114] A mapping relationship between a memory location and a disk location is established based on the memory mapping file technology to determine a target disk location corresponding to the target memory location.

[0115] In a possible implementation, the storage unit is specifically configured to:

[0116] During the low-peak period of daily network traffic, the temporary file is merged with the historical data file at the target disk location to obtain the target data file and the first index relationship.

[0117] In a possible implementation manner, the method further includes a temporary data unit, which is specifically used to:

[0118] The data set is stored in memory to obtain a third index relationship;

[0119] If a query request for a target attribute value is obtained, a data set corresponding to the target attribute value is obtained from the memory according to the third index relationship;

[0120] If the data set corresponding to the target attribute value fails to be obtained from the memory, the data set corresponding to the target attribute value is obtained from the disk location according to the first index relationship.

[0121] In a possible implementation, the acquiring unit is specifically configured to:

[0122] Obtain the data to be stored from the target process through an interface, middleware or application.

[0123] The names of the messages or information exchanged between multiple devices in the embodiments of the present application are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0124] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0125] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0126] Based on the above embodiments, an embodiment of the present application provides a system for preloading a list display object, characterized in that the system includes: a memory and a processor;

[0127] The memory is used to store program codes or instructions;

[0128] The processor is used to execute the program code or instruction in the memory to execute the method described in the above method embodiment.

[0129] Based on the above embodiments, an embodiment of the present application provides a computer-readable storage medium, characterized in that the computer storage medium stores program code or instructions, and when the program code or instructions are run on a computer, the computer executes the method described in the above method embodiments.

[0130] Based on the above embodiments, an embodiment of the present application provides a computer program product, characterized in that it includes a computer program or instructions; when the computer program or instructions are executed by a processor, the method described in the above method embodiments is executed.

[0131] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A data storage method based on attribute classification, characterized in that: The method comprises: Acquire data to be stored and a target attribute from a target process, wherein the data to be stored includes a plurality of sub-data, each of which has a plurality of attributes, and the target process indicates a target memory location where the data to be stored is stored; the target attribute is one of the plurality of attributes, and the target attribute is determined based on a query condition; Classifying the data to be stored according to the target attribute to obtain multiple data sets, wherein the sub-data in each data set has the same target attribute value; Determine the target disk position corresponding to the target memory position according to the mapping relationship between the memory position and the disk position, wherein the disk position is used to indicate the storage position of each sub-data in the data set in the file; If the target disk location has a target data file, multiple data sets are stored in corresponding storage locations in the target data file according to the target disk location to obtain a first index relationship, which is a correspondence between the target attribute value and the storage location of each sub-data in the corresponding data set in the target data file.

2. The method according to claim 1, characterized in that The step of storing the plurality of data sets in corresponding storage locations in the target data file according to the target disk locations to obtain a first index relationship includes: storing the plurality of data sets in storage locations corresponding to the temporary file according to the target disk locations, and obtaining a second index relationship, wherein the second index relationship is a correspondence relationship between the target attribute value and the storage location of the corresponding data set in the temporary file; The temporary file is merged with the historical data file at the target disk location to obtain the target data file and the first index relationship.

3. The method according to claim 1, characterized in that Determining the target disk position corresponding to the target memory position according to the mapping relationship between the memory position and the disk position includes: A mapping relationship between a memory location and a disk location is established according to a memory mapping file technology, and a target disk location corresponding to the target memory location is determined.

4. The method according to claim 2, characterized in that: The step of merging the temporary file with the historical data file at the target disk location to obtain the target data file and the first index relationship includes: During the daily low-peak period of network traffic, the temporary file is merged with the historical data file at the target disk location to obtain the target data file and the first index relationship.

5. The method according to claim 1, characterized in that The method further comprises: The data set is stored in the memory to obtain a third index relationship; If a query request for the target attribute value is obtained, obtaining a data set corresponding to the target attribute value from the memory according to the third index relationship; If the data set corresponding to the target attribute value fails to be obtained from the memory, the data set corresponding to the target attribute value is obtained from the disk location according to the first index relationship.

6. The method according to any one of claims 1 to 5, characterized in that: The obtaining of the data to be stored from the target process includes: Obtain the data to be stored from the target process through an interface, middleware or application.

7. A data storage device based on attribute classification, characterized in that: The device comprises: an acquisition unit, a classification unit, a positioning unit and a storage unit; The acquisition unit is used to acquire the data to be stored and the target attribute from the target process, the data to be stored includes a plurality of sub-data, each sub-data has a plurality of attributes, the target process indicates the target memory location where the data to be stored is stored; the target attribute is one of the plurality of attributes, and the target attribute is determined based on the query condition; The classification unit is used to classify the data to be stored according to the target attribute to obtain multiple data sets, and the sub-data in each data set has the same target attribute value; The positioning unit is used to determine the target disk position corresponding to the target memory position according to the mapping relationship between the memory position and the disk position, and the disk position is used to indicate the storage position of each sub-data in the data set in the file; The storage unit is used to store multiple data sets in corresponding storage positions in the target data file according to the target disk position if there is a target data file, so as to obtain a first index relationship, wherein the first index relationship is a correspondence between the target attribute value and the storage position of each sub-data in the corresponding data set in the target data file.

8. A computer device, characterized in that: The device comprises a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the method according to any one of claims 1 to 6 according to the instructions in the program code.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and the computer program is used to execute the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The method comprises a computer program or an instruction; when the computer program or the instruction is executed by a processor, the method according to any one of claims 1 to 6 is executed.