Data storage and database access method, device, electronic device and medium

Abstract

The present disclosure relates to a method, device, electronic device, and medium for data storage and database access, which can be used in a distributed database. The method includes: receiving a data storage request, the data storage request carrying a requesting user identifier and data to be stored; determining a corresponding target user data scale label based on the requesting user identifier; determining a target group routing algorithm corresponding to the target user data scale label based on data routing group configuration information; the data routing group configuration information is used to divide user data of different scales into data tables of different storage nodes of the database, so that the data volume load between each storage node is balanced; based on the target group routing algorithm, grouping and forwarding the data to be stored to the corresponding storage node, and constructing a storage mapping relationship between the data to be stored and the stored data table. This method realizes uniformly distributed storage at different user data volume levels and adapts to situations where the user data volume changes dynamically.

Description

Technical Field

[0001] The present disclosure relates to the field of distributed storage technology, and in particular to a method, device, electronic device, and medium for data storage and database access. Background Art

[0002] With the exponential growth of data in various applications and the emergence of high-concurrency access scenarios, the requirements for database capacity and access performance are becoming increasingly stringent. This has led to the emergence of distributed storage technology. The basic idea of ​​distributed storage is to disperse data from the original centralized database to multiple data storage nodes connected by a network to obtain larger storage capacity and higher concurrent access.

[0003] In the process of realizing the concept of the present disclosure, the inventors found that there are at least the following technical problems in the related art: when storing data based on a distributed database, most of the related technologies use user differences to divide the data into data tables of different storage nodes for storage. However, this division process will make the storage capacity of some data tables very large, and the storage capacity of some data tables small, resulting in serious data skew. In the process of application access to the database, it will also correspond to the problem of accessing a certain storage node in the hot data center; in the scenario where the volume of user data changes dynamically, this division method cannot be adapted to the situation where the volume of user data changes dynamically. Therefore, the storage logic in the related art has an adverse effect on the performance of the distributed database and the stability of the storage architecture when facing the situation where the data volume of different users varies greatly. Summary of the Invention

[0004] In order to solve the above technical problems or at least partially solve the above technical problems, embodiments of the present disclosure provide a method, device, electronic device, and medium for data storage and database access.

[0005] In a first aspect, an embodiment of the present disclosure provides a method for data storage. The method comprises: receiving a data storage request, the data storage request carrying a requesting user identifier and data to be stored; determining a corresponding target user data scale label based on the requesting user identifier; determining a target group routing algorithm corresponding to the target user data scale label based on data routing group configuration information; the data routing group configuration information is used to divide user data of different scales into data tables of different storage nodes in a database, so that the data level load between the storage nodes is balanced; grouping and forwarding the data to be stored to the corresponding storage node based on the target group routing algorithm, and establishing a storage mapping relationship between the data to be stored and the stored data table.

[0006] According to an embodiment of the present disclosure, the data routing group configuration information includes: a correspondence between a user data scale label and a group routing algorithm; the determining of a target group routing algorithm corresponding to the target user data scale label based on the data routing group configuration information includes: determining the target group routing algorithm corresponding to the target user data scale label based on the correspondence; when the target user data scale label is a user label of a first-level scale, the target group routing algorithm includes: performing a first routing grouping operation on attribute information of the data to be stored, so that the data to be stored is divided into a first number of data tables; when the target user data scale label is a user label of a second-level scale, the target group routing algorithm includes: performing a first routing grouping operation on attribute information of the data to be stored, obtaining a first logical grouping result; and performing a second routing grouping operation on the first logical grouping result, so that the data to be stored is divided into a second number of data tables; wherein the second-level scale is larger than the first-level scale, and the second number is larger than the first number.

[0007] According to an embodiment of the present disclosure, the first routing grouping operation includes: a hash modulo operation; the second routing grouping operation includes: performing grouping operations according to a preset time period; and the attribute information includes: a requesting user identifier.

[0008] According to an embodiment of the present disclosure, determining the corresponding target user data scale tag based on the above-mentioned requesting user identifier includes: matching in a pre-configured user tag configuration table based on the above-mentioned requesting user identifier to obtain a matching user data scale tag; the above-mentioned user tag configuration table includes: user identifier, user data scale tag and tag time; the above-mentioned tag time is used to indicate the valid time of the user data scale tag; and determining the target user data scale tag corresponding to the reception time of the above-mentioned data storage request based on the matching tag time corresponding to the above-mentioned matching user data scale tag.

[0009] According to an embodiment of the present disclosure, the tag time is represented by the start time of the user data scale tag, which is used to indicate that the time period from the previous start time to the next start time is valid. According to the matching tag time corresponding to the matching user data scale tag, the target user data scale tag corresponding to the reception time of the data storage request is determined, including: when the matching tag time includes multiple matching start times and all of them are earlier than the reception time, the matching user data scale tag corresponding to the latest matching start time among the multiple matching start times is determined as the target user data scale tag; when the matching tag time includes multiple matching start times and the reception time falls within the time period between the first matching start time and the second matching start time, the matching user data scale tag corresponding to the first matching start time is determined as the target user data scale tag.

[0010] According to an embodiment of the present disclosure, the above-mentioned tag time is represented by the duration information of the user data scale tag; wherein, based on the matching tag time corresponding to the above-mentioned matching user data scale tag, the target user data scale tag corresponding to the reception moment of the above-mentioned data storage request is determined, including: when the above-mentioned matching tag time contains one or more matching duration information, determining the target matching duration information into which the above-mentioned reception moment falls; and determining the matching user data scale tag corresponding to the above-mentioned target matching duration information as the above-mentioned target user data scale tag.

[0011] According to an embodiment of the present disclosure, the above-mentioned data storage method also includes: generating scale label information within the above-mentioned statistical period for the above-mentioned target user based on at least one of the stored data and database access performance of the target user within the statistical period; and generating or updating the user data scale label in the above-mentioned user label configuration table based on the above-mentioned scale label information.

[0012] According to an embodiment of the present disclosure, the above-mentioned data storage method is applied to a storage service end including a database middleware or a storage demand end including a database middleware. Among them, according to the above-mentioned target group routing algorithm, the above-mentioned data to be stored are grouped and forwarded to the corresponding storage node, and a storage mapping relationship between the above-mentioned data to be stored and the stored data table is constructed, including: executing the above-mentioned target group routing algorithm on the attribute information of the above-mentioned data to be stored, dividing the above-mentioned data to be stored into the corresponding data tables of different storage nodes of the above-mentioned database; constructing an association relationship between the data to be stored and the stored data table; according to the request user identifier of the above-mentioned data to be stored, the target user data scale label and the corresponding label time, the above-mentioned association relationship is associated and expanded to obtain a multi-dimensional storage mapping relationship; the above-mentioned storage mapping relationship includes: the association relationship between the user identifier, the user data scale label, the label time and the stored data table.

[0013] In a second aspect, embodiments of the present disclosure provide a data access method. This method, applied to a storage demand side including database middleware, comprises: receiving a data access request, the data access request carrying: an access user identifier and a data operation instruction for the data to be accessed; the data operation instruction comprising at least one of the following: a query instruction, a modification instruction, and a deletion instruction; determining a corresponding access user data scale tag based on the access user identifier; querying a target data table corresponding to the access user identifier and the access user data scale tag based on a pre-established storage mapping relationship; and forwarding the data operation instruction for the data to be accessed in the target data table to a corresponding storage node.

[0014] In a third aspect, embodiments of the present disclosure provide a data access method. This method, applied to a storage service end including database middleware, comprises: receiving a data access request, the data access request carrying: an access user identifier and a data operation instruction for the data to be accessed; the data operation instruction comprising at least one of the following: a query instruction, a modification instruction, and a deletion instruction; determining a corresponding access user data scale tag based on the access user identifier; querying a target data table corresponding to the access user identifier and the access user data scale tag based on a pre-established storage mapping relationship; and executing the data operation instruction on the data to be accessed in the target data table to obtain a data operation result.

[0015] According to an embodiment of the present disclosure, in the data access method provided in the second or third aspect, the data access request further carries: a search period for the data to be accessed; the storage mapping relationship includes: an association relationship between the user identifier, the user data scale tag, the tag time, and the stored data table; the tag time is used to indicate the valid time of the user data scale tag. According to the pre-established mapping relationship, querying the target data table corresponding to the access user identifier and the access user data scale tag includes: filtering the target tag time that matches the search period in the tag time of the mapping storage relationship; and querying the target data table corresponding to the access user identifier and the access user data scale tag within the limited time period of the target tag time.

[0016] In a fourth aspect, an embodiment of the present disclosure provides a device for data storage. The above-mentioned device is a storage demand end including a database middleware or a storage service end including a database middleware, and includes: a storage request receiving module, a storage label determination module, a routing algorithm determination module and a group routing module. The above-mentioned storage request receiving module is used to receive a data storage request, and the above-mentioned data storage request carries a requesting user identifier and data to be stored. The above-mentioned storage label determination module is used to determine the corresponding target user data scale label according to the above-mentioned requesting user identifier. The above-mentioned routing algorithm determination module is used to determine the target group routing algorithm corresponding to the above-mentioned target user data scale label according to the data routing group configuration information; the above-mentioned data routing group configuration information is used to divide user data of different scales into data tables of different storage nodes of the database, so that the data level load between each storage node is balanced. The above-mentioned group routing module is used to group and forward the above-mentioned data to be stored to the corresponding storage node according to the above-mentioned target group routing algorithm, and construct a storage mapping relationship between the above-mentioned data to be stored and the stored data table.

[0017] In a fifth aspect, an embodiment of the present disclosure provides a device for data access. The above-mentioned device is a storage demand end including a database middleware, and includes: a first access request receiving module, a first access label determination module, a first storage table query module and a forwarding module. The above-mentioned first access request receiving module is used to receive a data access request, and the above-mentioned data access request carries: an access user identifier and a data operation instruction for the data to be accessed; the above-mentioned data operation instruction includes at least one of the following: a query instruction, a modification instruction, and a deletion instruction. The above-mentioned first access label determination module is used to determine the corresponding access user data scale label based on the above-mentioned access user identifier. The above-mentioned first storage table query module is used to query the target data table corresponding to the above-mentioned access user identifier and the above-mentioned access user data scale label based on a pre-constructed storage mapping relationship. The above-mentioned forwarding module is used to forward the data operation instruction for the data to be accessed in the target data table to the corresponding storage node.

[0018] In the sixth aspect, an embodiment of the present disclosure provides a device for data access. The above-mentioned device is a storage service end including a database middleware, and includes: a second access request receiving module, a second access label determination module, a second storage table query module and a data operation module. The above-mentioned second access request receiving module is used to receive a data access request, and the above-mentioned data access request carries: an access user identifier and a data operation instruction for the data to be accessed; the above-mentioned data operation instruction includes at least one of the following: a query instruction, a modification instruction, and a deletion instruction. The above-mentioned second access label determination module is used to determine the corresponding access user data scale label based on the above-mentioned access user identifier. The above-mentioned second storage table query module is used to query the target data table corresponding to the above-mentioned access user identifier and the above-mentioned access user data scale label based on a pre-constructed storage mapping relationship. The above-mentioned data operation module is used to execute the above-mentioned data operation instruction on the data to be accessed in the above-mentioned target data table to obtain a data operation result.

[0019] In a seventh aspect, embodiments of the present disclosure provide an electronic device. The electronic device includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus; the memory is configured to store computer programs; and the processor is configured to implement the aforementioned data storage method or data access method when executing the program stored in the memory.

[0020] In an eighth aspect, embodiments of the present disclosure provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the data storage method or data access method described above.

[0021] The above technical solutions provided by the embodiments of the present disclosure have at least some or all of the following advantages:

[0022] Since different users have their own user data size labels, different group routing algorithms are configured for different user data size labels in the data routing group configuration information. After receiving a data storage request, the corresponding target user data size label is determined based on the requesting user identifier, and the target group routing algorithm corresponding to the target user data size label is determined based on the data routing group configuration information. The corresponding target group routing algorithm is executed accordingly, so that user data of different sizes can achieve data volume load balancing among the storage nodes in the process of dividing the data tables corresponding to the storage nodes of the database. On the one hand, uniform group storage is achieved for different user data volume levels, avoiding large differences in the data storage volume between different storage nodes, effectively solving the problem of data skew, and also effectively avoiding the impact of access concentration caused by the concentration of hot data on a certain storage node on database stability. On the other hand, when the user data volume changes dynamically, the corresponding user data size label and the corresponding group routing algorithm will be reflected in the change, so that the storage and access are balanced accordingly to the change in the user data volume, thus achieving dynamic adaptive storage and access. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0024] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0025] Figure 1A A system architecture of a data storage method and a database access method applicable to an embodiment of the present disclosure is schematically shown;

[0026] Figure 1B Schematically illustrates another system architecture applicable to the data storage method and database access method of the embodiment of the present disclosure;

[0027] Figure 2 The following schematically shows a flow chart of a data storage method according to an embodiment of the present disclosure;

[0028] Figure 3 Schematically shows a detailed implementation flow chart of step S220 according to an embodiment of the present disclosure;

[0029] Figure 4Schematically shows a detailed implementation flow chart of step S320 according to an embodiment of the present disclosure;

[0030] Figure 5 Schematically shows a detailed implementation flow chart of step S240 according to an embodiment of the present disclosure;

[0031] Figure 6 The following schematically shows a flow chart of a method for data access applied to a storage demand side according to an embodiment of the present disclosure;

[0032] Figure 7 The following schematically shows a flow chart of a method for data access applied to a storage service client according to an embodiment of the present disclosure;

[0033] Figure 8 The following schematically shows a structural block diagram of a data storage device according to an embodiment of the present disclosure;

[0034] Figure 9 The following schematically shows a structural block diagram of a data access device according to an embodiment of the present disclosure;

[0035] Figure 10 The following schematically shows a structural block diagram of a data access device according to another embodiment of the present disclosure;

[0036] Figure 11 The structural block diagram of the electronic device provided by the embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0037] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure more clear, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present disclosure without making any creative efforts shall fall within the scope of protection of the present disclosure.

[0038] Figure 1A A system architecture of a data storage method and a database access method applicable to an embodiment of the present disclosure is schematically shown; Figure 1B Another system architecture of the data storage method and database access method applicable to the embodiments of the present disclosure is schematically shown.

[0039] Reference Figure 1A and Figure 1B As shown, the system architecture of the data storage method and database access method applicable to the embodiments of the present disclosure includes: a storage demand end 110 and a storage service end 120.

[0040] The storage demand end 110 may be various devices with data storage requirements, such as terminal devices with various applications 111 installed or servers with data storage requirements (such as servers that provide data computing service support for the applications 111).

[0041] The above-mentioned applications 111 include but are not limited to: shopping applications, web browser applications, search applications, short video applications, video playback applications, image editing applications, instant messaging tools, email clients, social platform software, etc.

[0042] The storage service end 120 is used to provide data storage services for the storage demand end 110. In some embodiments, the storage service end 120 may have its own database or be able to access a database.

[0043] Figure 1A Example system architecture 100a and Figure 1B The differences between the example system architecture 100b are: the difference in the proxy mode of the database middleware, Figure 1A The example is the server proxy mode, Figure 1B The example is a client proxy mode; in order to distinguish and describe different proxy mode architectures, the first database middleware and the second database middleware are used to distinguish and describe the database middleware under different proxy mode architectures.

[0044] Reference Figure 1A As shown, in some embodiments, under the system architecture 100a corresponding to the server-side proxy mode, the storage service end 120 includes a first database middleware 121. The first database middleware 121 is located between the underlying database and the upper-layer application and is used to shield the underlying details of each database (for example, a heterogeneous database) in the distributed database. The first database middleware 121 can serve as a bridge for communication between each application 111 in the storage demand end 110 and the distributed database.

[0045] The distributed database includes multiple storage nodes, for example, database A1, database A2, and database A3.

[0046] Reference Figure 1B As shown, in other embodiments, under the system architecture 100b corresponding to the client proxy mode, the storage demand end 110 includes a second database middleware 112. The second database middleware 112 is located between the underlying database and the upper-layer application and is used to shield the underlying details of each database in the distributed database (for example, a heterogeneous database). The second database middleware 112 can serve as a bridge for communication between each application 111 in the storage demand end 110 and the distributed database. Figure 1BAs shown, in the system architecture 100b corresponding to the client proxy mode, the second database middleware 112 includes multiple connection pools for respectively interacting with the databases to which they are connected. For example, Figure 1B In the example, connection pools 1 to 3 are included. Connection pool 1 is used to connect to and interact with database A1 in the distributed database. Connection pool 2 is used to connect to and interact with database A2 in the distributed database. Connection pool 3 is used to connect to and interact with database A3 in the distributed database.

[0047] The data storage method provided in the embodiments of the present disclosure can be executed by the storage service end 120 including the first database middleware 121 in the system architecture 100a, or by the storage demand end 110 including the second database middleware 112 in the system architecture 100b. Accordingly, the data access method provided in the embodiments of the present disclosure can be executed by the storage service end 120 including the first database middleware 121 in the system architecture 100a, or by the storage demand end 110 including the second database middleware 112 in the system architecture 100b. This will be described in detail below with reference to specific embodiments.

[0048] The first exemplary embodiment of the present disclosure provides a method for storing data. The method of this embodiment can be applied to Figure 1A The system architecture 100a includes a storage service end 120 of the first database middleware 121; or applied to Figure 1B The system architecture 100 b includes a storage demand end 110 of a second database middleware 112 .

[0049] Figure 2 The flowchart schematically shows a data storage method according to an embodiment of the present disclosure.

[0050] Reference Figure 2 As shown, the data storage method provided by the embodiment of the present disclosure includes the following steps: S210, S220, S230 and S240.

[0051] In step S210, a data storage request is received, where the data storage request carries a requesting user identifier and data to be stored.

[0052] In one implementation scenario, refer to Figure 1BAs shown, taking a shopping application as an example, a user (e.g., an individual user, a merchant user, etc.) generates business data by operating application 111. According to the business logic, some business data is automatically saved to the database (e.g., the user's browsing history in the shopping application) or a data storage request is initiated to the database under the user's save instruction. Accordingly, the storage demand end 110 receives the above data storage request. The second database middleware 112 of the storage demand end 110 executes the subsequent steps S220 to S240 accordingly.

[0053] In other implementation scenarios, refer to Figure 1A As shown, the storage demand end 110 forwards the data storage request of the application 111 to the storage service end 120. Accordingly, the storage service end 120 receives the data storage request, and the first database middleware 121 of the storage service end 120 executes the subsequent steps S220 to S240.

[0054] In step S220, the corresponding target user data size label is determined according to the requesting user identifier.

[0055] In some embodiments, a user data size tag corresponding to each user ID is generated in advance in the execution entity (e.g., the storage demand end 110 or the storage service end 120) based on at least one of the historical stored data of different users and the database access performance. For example, the user data size tag of each user can be stored in a user tag configuration table.

[0056] For example, the above-mentioned data storage method also includes the step of pre-generating a user data scale label, which specifically includes the following execution steps: generating scale label information within the above-mentioned statistical period for the above-mentioned target user based on the stored data and database access performance of the target user within the statistical period; generating or updating the user data scale label in the above-mentioned user label configuration table based on the above-mentioned scale label information.

[0057] For example, the classification labels for user data scale corresponding to different users can be determined based on at least one of the existing stored data and the current database access performance. If a user's business data exceeds a set threshold (e.g., 100 million) or exceeds a certain level, the user is assigned a large-scale label. If a user's business data is less than a set threshold or within a certain data level, the user is assigned a small-scale label. Alternatively, if the database server's disk space usage exceeds a set ratio (e.g., 85% to 90%) or database access performance fluctuates, performance monitoring and diagnosis (e.g., through business-level metric monitoring or database-level performance monitoring) can locate the cause of data access skew, determine the specific user ID causing the data skew, and assign the large-scale label to that specific user ID. It will be understood that while large-scale and small-scale labels are used as examples here, labels of different scales can actually be divided into more levels, with user labels of two or more levels corresponding to different packet routing algorithms.

[0058] In other embodiments, considering that the user's data volume changes dynamically over time, the corresponding user data scale label will also change accordingly. Therefore, a label time field is also set in the user label configuration table. The label time is used to indicate the effective time of the user data scale label.

[0059] In step S230, a target group routing algorithm corresponding to the target user data scale label is determined based on the data routing group configuration information; the data routing group configuration information is used to divide user data of different scales into data tables of different storage nodes of the database, so that the data scale load between each storage node is balanced.

[0060] In some embodiments, the data routing group configuration information includes: a correspondence between a user data size tag and a group routing algorithm. The group routing algorithm and the correspondence may be pre-set by system designers, operation and maintenance personnel, etc. of the data storage service and may be modified and updated.

[0061] The determining of the target group routing algorithm corresponding to the target user data scale label according to the data routing group configuration information includes: determining the target group routing algorithm corresponding to the target user data scale label according to the corresponding relationship.

[0062] In some embodiments, when the target user data scale label is a first-level scale user label, the target group routing algorithm includes: performing a first routing grouping operation on the attribute information of the data to be stored, so that the data to be stored is divided into a first number of data tables.

[0063] In some embodiments, when the target user data scale label is a user label of the second-level scale, the target group routing algorithm includes: performing a first routing grouping operation on the attribute information of the data to be stored to obtain a first logical grouping result; performing a second routing grouping operation on the first logical grouping result, so that the data to be stored is divided into a second number of data tables; wherein the second-level scale is larger than the first-level scale, and the second number is larger than the first number.

[0064] The above attribute information may cover various description dimensions of the data to be stored, including but not limited to one of the following attributes: the requesting user identifier of the data to be stored, the generation timestamp of the data to be stored, the size of the storage space occupied by the data to be stored, etc.

[0065] In some embodiments, the first routing grouping operation may be a hash modulus operation or other operations capable of relatively uniform grouping. The second routing grouping operation may be a grouping operation performed according to a preset time period or a grouping operation in a different operation dimension from the first routing grouping operation.

[0066] In some embodiments, the meaning of data level load balancing between storage nodes can be: the data to be stored is divided into M groups, and these M groups of data are used to store in the corresponding data tables of K storage nodes respectively. The order of magnitude of the stored data in each storage node is equal within the required accuracy, and M and K are both integers ≥2.

[0067] In step S240, the data to be stored are grouped and forwarded to corresponding storage nodes according to the target group routing algorithm, and a storage mapping relationship between the data to be stored and the stored data table is constructed.

[0068] Based on the target group routing algorithm, the data to be stored can be divided into multiple groups, and the data of each group corresponds to the corresponding data table of the corresponding storage node.

[0069] For example, user identification P1 is identified as a small user tag during the time period T1 to T2, and the corresponding data volume X1 to be stored is 3G (the specific value and magnitude are only for example), among which 0-1G (left-open and closed interval, the subsequent representations in this paragraph have the same meaning) of data are divided into data table B11 of database A1, 1G-2G of data are divided into data table B21 of database A2, and 2G-3G of data are divided into data table B31 of database A3.

[0070] A hash modulo operation is performed on the attribute information of the data to be stored (an example of the first routing grouping operation), including: processing the above-mentioned attribute information based on the hash function to obtain an output value of a fixed length; then performing a modulo operation on the output value (as the dividend) and a target value (as the divisor) (when the dividend and the divisor are both positive numbers, the modulo operation is equivalent to the remainder operation), and the obtained remainder corresponds to the group number of the division, and the target value uses the total number of groups to be divided (for example, the first number), and the group number of the division has a corresponding relationship with the data table in each storage node.

[0071] For example, for the data to be stored with small-scale tags, assuming that the first number is 3, the data to be stored is divided into 3 groups by performing the first routing grouping operation. The data level of each group of data is similar and corresponds to 3 storage nodes (for example Figure 1A and Figure 1B In one storage node of the example databases A1 to A3), each storage node uses a corresponding data table to store grouped data.

[0072] For large-scale labeled data to be stored, assuming that the second number is 12, the data to be stored is divided into 3 groups by performing the first routing grouping operation, and 3 logical grouping results are obtained; then the data is divided based on the time period in which each group of data is located (an example of the second routing grouping operation). For example, data of the same quarter are divided into one group, and each group of the above 3 logical grouping results is further divided into 4 groups, resulting in a total of 12 groups. The data of these 12 groups are stored in 3 storage nodes respectively (for example Figure 1A and Figure 1B In the example databases A1 to A3), each storage node uses four corresponding data tables to store grouped data.

[0073] In some embodiments, the storage mapping relationship is used to represent the mapping relationship between the index information of the data to be stored and the stored data table, and the storage mapping relationship is used to locate the data storage location during the data access process.

[0074] In some embodiments, the index information includes: a user identifier and a user data size tag.

[0075] In some embodiments, given that the user's data volume changes dynamically over time, the corresponding user data size tag will also change accordingly. Therefore, a tag time field is also provided in the user tag configuration table. The tag time is used to indicate the validity period of the user data size tag. The index information includes: user ID, user data size tag, and tag time.

[0076] For example, in some implementation scenarios, since the actual data volume of the user identifier P1 changes over time, the corresponding user data scale label will also change accordingly; assuming that the user identifier P1 is identified as a large user label after time T2, the corresponding data volume of the data X2 to be stored is 30G, which is divided into 30 data tables through two-dimensional group routing operations. For example, 10 groups of data of 0~1G, 1G~2G, 2G~3G, 3G~4G, 4G~5G, 5G~6G, 6G~7G, 7G~8G, 8G~9G, and 9G~10G are respectively divided into data tables B101, B102, B103, B104, B105, B106, B107, B108, B109, and B110 of database A1; 10G~11G, 11G~12G, and 12G~13G are respectively divided into data tables B101, B102, B103, B104, B105, B106, B107, B108, B109, and B110 of database A1; The 10 groups of data of 1G, 13G~14G, 14G~15G, 15G~16G, 16G~17G, 17G~18G, 18G~19G, and 19G~20G are respectively divided into data tables B201, B202, B203, B204, B205, B206, B207, B208, B209, and B210 of database A2; 20G~21G, 21G The 10 groups of data, 22G~23G, 23G~24G, 24G~25G, 25G~26G, 26G~27G, 27G~28G, 28G~29G, and 29G~30G, are respectively divided into data tables B301, B302, B303, B304, B305, B306, B307, B308, B309, and B310 of database A3.

[0077] It should be understood that the above embodiment assumes that the storage capacity of each data table is the same. In reality, the amount of data stored in each data table can vary, as long as the data level distribution between each storage node is uniform. Uniform data level distribution has more relaxed restrictions than uniform data volume distribution, allowing for slight differences in data volume between storage nodes. It also helps to balance the load between storage nodes during data access.

[0078] In the embodiment including steps S210 to S240, since different users have their own corresponding user data size labels, different group routing algorithms are configured for different user data size labels in the data routing group configuration information. After receiving a data storage request, the corresponding target user data size label is determined based on the requesting user identifier, and the target group routing algorithm corresponding to the target user data size label is determined based on the data routing group configuration information. The corresponding target group routing algorithm is executed accordingly, so that user data of different sizes is load balanced among the storage nodes in the process of dividing the data tables corresponding to the storage nodes of the database. On the one hand, uniform group storage is achieved for different user data volume levels, avoiding large differences in the data storage volume between different storage nodes, and effectively solving the problem of data skew. On the other hand, when the user data volume changes dynamically, the corresponding user data size label and the corresponding group routing algorithm will also be reflected in the changes, so that the storage is balanced accordingly to dynamically adapt to the changes in the user data volume, thereby achieving dynamic adaptive storage.

[0079] Figure 3 The detailed implementation flow chart of step S220 according to an embodiment of the present disclosure is schematically shown.

[0080] Reference Figure 3 As shown, according to an embodiment of the present disclosure, in the above step S220, the corresponding target user data size label is determined according to the above requesting user identifier, including the following steps: S310 and S320.

[0081] In step S310, based on the requesting user identifier, a match is performed in a pre-configured user tag configuration table to obtain a matching user data scale tag; the user tag configuration table includes: user identifier, user data scale tag and tag time; the tag time is used to indicate the valid time of the user data scale tag.

[0082] The tag time can be represented in various forms. For example, in some embodiments, the tag time is represented by the start time (also understood as the change time) of the user data size tag, indicating that the time period from the previous start time to the next start time is valid. In other embodiments, the tag time is represented by the duration information of the user data size tag.

[0083] For example, the user tag configuration table is as shown in Table 1 below:

[0084]

[0085] In the above Table 1, label 0 is used to represent small-scale user data, and label 1 is used to represent large-scale user data.

[0086] The user tag configuration table also needs to be dynamically maintained as time changes. For example, if the data scale of a certain user changes, adaptability changes or new user data scale tags need to be added.

[0087] For example, the updated user tag configuration table is as shown in Table 2 below:

[0088]

[0089] Referring to Table 2, it can be seen that for the user identified as abc5004, from August 1, 2022 to May 1, 2023, the user data scale label of user abc5004 is a small-scale label (corresponding label value is 0). Starting from May 1, 2023, the user data scale label of user abc5004 changes to a large-scale label (corresponding label value is 1).

[0090] In step S320, the target user data size tag corresponding to the reception time of the data storage request is determined according to the matching tag time corresponding to the matching user data size tag.

[0091] Figure 4 The detailed implementation flow chart of step S320 according to an embodiment of the present disclosure is schematically shown.

[0092] In some embodiments, reference Figure 4 As shown, the tag time is represented by the start time of the user data size tag, indicating the validity period from the previous start time to the next start time. With this representation, only one transition time and the corresponding user data size tag after the transition need to be recorded in the user tag configuration table, which saves storage space in the user tag configuration table and simplifies the field storage format.

[0093] In the above step S320, determining the target user data size tag corresponding to the reception time of the above data storage request according to the matching tag time corresponding to the above matching user data size tag includes the following steps: S410a or S410b.

[0094] In step S410a, when the matching tag time includes multiple matching start times and all of them are earlier than the receiving time, the matching user data scale tag corresponding to the latest matching start time among the multiple matching start times is determined as the target user data scale tag.

[0095] Reference Figure 4As shown, a requesting user identifier corresponds to multiple matching user data size tags, each of which has a corresponding matching tag time. For example, in the example, matching start time t11 and matching start time t12, t12 is later than t11. From matching start time t11 until matching start time t12, the tag value of the matching user data size tag is 0, i.e., a small user tag. Starting from matching start time t12, the tag value changes to 1, i.e., the matching user data size tag changes to a large user tag. If multiple matching start times are all earlier than receiving time t0, the matching user data size tag corresponding to the latest matching start time t12 among the multiple matching start times, i.e., the large user tag, is determined as the target user data size tag corresponding to receiving time t0.

[0096] In step S410b, when the matching tag time includes multiple matching start times and the receiving time falls within the period between the first matching start time and the second matching start time, the matching user data scale tag corresponding to the first matching start time is determined as the target user data scale tag.

[0097] Reference Figure 4 As shown, a requesting user identifier corresponds to multiple matching user data size tags, each of which has corresponding matching tag times, such as matching start time t11, matching start time t12, and matching start time t13, where t12 is later than t11 and t13 is later than t12. From matching start time t11 until matching start time t12, the tag value of the matching user data size tag is 0, i.e., a small user tag. From matching start time t12 until matching start time t13, the tag value of the matching user data size tag changes to 1, i.e., the matching user data size tag changes to a large user tag. After matching start time t13, the tag value of the matching user data size tag changes back to 0, i.e., a small user tag.

[0098] For the case where the above-mentioned receiving time t0 falls within the time period between t12 (as an example of the first matching start time) and t13 (as an example of the second matching start time), the matching user data scale label (for example, the large user label) corresponding to the above-mentioned first matching start time t12 is determined as the target user data scale label corresponding to the receiving time t0.

[0099] In other embodiments, the above-mentioned label time is represented by the duration information of the user data scale label. In the representation method of this embodiment, since it is difficult to predict the change of the user scale, the current moment is used as the end moment of the label within any valid moment. For example, referring to the illustration in Table 3: when there is no label change, the corresponding duration information is expressed in the form of: starting moment-current moment. As the current moment passes, the end moment of the duration information changes dynamically. When the user data scale label changes, the end moment of the duration is updated to the final change moment. Therefore, the duration information here is constantly changing and updated over time.

[0100] For example, using this representation, the example for the situation in Table 2 is as follows:

[0101]

[0102] Among them, in the above-mentioned step S320, based on the matching tag time corresponding to the above-mentioned matching user data scale tag, the target user data scale tag corresponding to the reception moment of the above-mentioned data storage request is determined, including: when the above-mentioned matching tag time contains one or more matching duration information, determining the target matching duration information into which the above-mentioned reception moment falls; and determining the matching user data scale tag corresponding to the above-mentioned target matching duration information as the above-mentioned target user data scale tag.

[0103] Figure 5 The detailed implementation flow chart of step S240 according to an embodiment of the present disclosure is schematically shown.

[0104] Reference Figure 5 As shown, in some embodiments, in the above step S240, according to the above target group routing algorithm, the above data to be stored is grouped and forwarded to the corresponding storage node, and a storage mapping relationship between the above data to be stored and the stored data table is constructed, including the following steps: S510, S520 and S530.

[0105] In step S510, the target group routing algorithm is executed on the attribute information of the data to be stored, and the data to be stored are grouped and divided into corresponding data tables of different storage nodes of the database.

[0106] Reference Figure 1BAs shown, when the execution subject is the storage demand side 110, the second database middleware 112 executes the target group routing algorithm to divide the data to be stored corresponding to different user data size tags into different groups, and each group of data corresponds to the corresponding data table of the corresponding storage node. For example, for the data to be stored for user ID P1 in the time period T1 to T2, the storage demand side 110 implements the division of each group of data and the corresponding data table. Afterwards, the storage demand side 110 can send each group of data to the corresponding database A1 to A3 through each connection pool 1 to 3 of the second database middleware 112. For example, through connection pool 1, 0 to 1G (left open and closed interval, the subsequent representations in this paragraph have the same meaning) of the data to be stored is sent to database A1, requesting that it be stored in data table B11; through connection pool 2, 1G to 2G of the data to be stored is sent to database A2, requesting that it be stored in data table B21; and through connection pool 3, 2G to 3G of the data to be stored is sent to database A3, requesting that it be stored in data table B31.

[0107] Reference Figure 1A As shown, when the execution subject is the storage service end 120, the first database middleware 121 divides the data to be stored corresponding to different user data scale labels into groups to different degrees by executing the target group routing algorithm, and each group of data corresponds to the corresponding data table of the corresponding storage node. For the data to be stored in the time period T1 to T2 for user identifier P1, the storage service end 120 realizes the division of each group of data and the corresponding data table. Afterwards, the storage service end 120 can send each group of data to the corresponding database A1 to A3 through the first database middleware 121. The corresponding data is stored in the corresponding data tables B11 to B31 of each database A1 to A3.

[0108] In step S520, an association relationship between the data to be stored and the stored data table is constructed.

[0109] For example, the constructed association relationship is: {data to be stored 0~1G data → (arrows are used here to represent association relationships) data table B11 of database A1, data to be stored 1G~2G data → data table B21 of database A2, data to be stored 2G~3G data → data table B31 of database A3}.

[0110] In step S530, the association relationship is expanded based on the requesting user ID, the target user data size tag, and the corresponding tag time of the data to be stored, thereby obtaining a multi-dimensional storage mapping relationship. The storage mapping relationship includes the association relationship between the user ID, the user data size tag, the tag time, and the stored data table. The tag time indicates the validity period of the user data size tag.

[0111] After the association expansion, the storage mapping relationship obtained is: {P1→label 0→label time (valid time of label 0)→data table B11 of database A1, P1→label 0 (valid time of label 0)→label time→data table B21 of database A2, P1→label 0→label time (valid time of label 0)→data table B31 of database A3}.

[0112] The above-mentioned storage mapping relationship is used to represent the mapping relationship between the index information of the data to be stored and the stored data table, and the above-mentioned storage mapping relationship is used to locate the data storage location during data access (broadly speaking, access means, including data query, data modification, data deletion and other operations). This embodiment takes the index information including: user identification, user data scale label and label time as an example. In this embodiment, due to the parameter containing the dimension of label time, the solution can accurately locate the stored data of each historical period, support the setting of filtering period for data query, and for the situation where the routing logic of the stored data table is different due to the change of data volume for the same user, by constructing a storage mapping relationship including label time, during the process of data access, it can be matched based on the label time in the index information and the filtering period set during the access, thereby locating the corresponding target data table under the group routing algorithm of the filtering period.

[0113] In the solution of the storage mapping relationship without the tag period, it can adapt to the situation where the volume of user data remains unchanged and realize the access of the stored data of each user.

[0114] Based on the same concept, the second exemplary embodiment of the present disclosure provides a data access method, which is applied to a storage demand side including a database middleware. Figure 1B The exemplary system architecture 100b is combined with the solution in the first embodiment in which the execution subject is the storage demand end 110 including the second database middleware 112 to understand the content of this embodiment.

[0115] Figure 6 A flowchart of a method for data access applied to a storage demand side according to an embodiment of the present disclosure is schematically shown.

[0116] Combine Figure 1B and Figure 6 As shown, the method for data access applied to the storage demand side provided by the embodiment of the present disclosure includes the following steps: S610, S620, S630 and S640.

[0117] In step S610, a data access request is received, where the data access request carries: an access user identifier and a data operation instruction for the data to be accessed.

[0118] The above-mentioned data operation instruction includes at least one of the following: a query instruction, a modification instruction, and a deletion instruction.

[0119] In step S620, the corresponding access user data scale label is determined according to the access user identifier.

[0120] The specific implementation process of this embodiment can refer to the detailed description of step S220 in the first embodiment mentioned above. The entire implementation process of this step can be incorporated into this embodiment. The only difference is that one is a data storage process and the other is a data access process. The requesting user identifier in the data storage process is replaced by the accessing user identifier in the data access process, and the result is replaced by the accessing user data scale label.

[0121] In step S630, according to the pre-built storage mapping relationship, the target data table corresponding to the access user identifier and the access user data size tag is searched.

[0122] In some embodiments, the storage mapping relationship includes: an association relationship between a user identifier, a user data size tag, and a stored data table.

[0123] In other embodiments, the storage mapping relationship includes: an association relationship between a user identifier, a user data size tag, a tag time and a stored data table, wherein the tag time is used to indicate a valid time of the user data size tag.

[0124] The data access request also includes a search period for the data to be accessed. In step S630, based on the pre-established storage mapping relationship, querying the target data table corresponding to the access user identifier and the access user data scale tag includes: selecting a target tag time that matches the search period from the tag time in the mapping storage relationship; and querying the target data table corresponding to the access user identifier and the access user data scale tag within the limited period of the target tag time.

[0125] The search period can be used to search for data of one or more time periods, or it can be used to search for data of one or more time points.

[0126] In step S640 , the data operation instruction for the data to be accessed in the target data table is forwarded to the corresponding storage node.

[0127] Reference Figure 1B As shown, the above data access method may further include: receiving data operation results fed back by each storage node.

[0128] The data access method provided in this embodiment achieves load balancing of the stored data volume among various distributed storage nodes, thereby reducing the occurrence of data skew in distributed databases and effectively avoiding the situation where data access hotspots are concentrated on a certain storage node. During the data access process, not only is the adverse impact of hotspot access concentrated on a certain storage node on database stability avoided, but also, when the user's data volume changes dynamically, the corresponding user data scale label and the corresponding group routing algorithm will be reflected in the change, thereby dynamically adapting to the change in user data volume and performing corresponding balanced storage and access, thereby achieving dynamic adaptive storage and access.

[0129] The third exemplary embodiment of the present disclosure provides a method for data access. The method is applied to a storage service end including a database middleware. Figure 1A The exemplary system architecture 100a is combined with the solution in the first embodiment in which the execution subject is the storage service end 120 including the first database middleware 121 to understand the content of this embodiment. The difference between this embodiment and the second embodiment lies in the difference in the execution subject. Accordingly, there is a difference between step S640 and step S740.

[0130] Figure 7 A flowchart of a method for data access applied to a storage service client according to an embodiment of the present disclosure is schematically shown.

[0131] Reference Figure 7 As shown, the data access method applied to the storage service end provided by the embodiment of the present disclosure includes the following steps: S710, S720, S730 and S740.

[0132] In step S710, a data access request is received, wherein the data access request carries: an access user identifier and a data operation instruction for the data to be accessed, wherein the data operation instruction includes at least one of the following: a query instruction, a modification instruction, and a deletion instruction.

[0133] In step S720, the corresponding access user data size label is determined according to the access user identifier.

[0134] The specific implementation process of this embodiment can refer to the detailed description of step S220 in the first embodiment mentioned above. The entire implementation process of this step can be incorporated into this embodiment. The only difference is that one is a data storage process and the other is a data access process. The requesting user identifier in the data storage process is replaced by the accessing user identifier in the data access process, and the result is replaced by the accessing user data scale label.

[0135] In step S730, according to the pre-built storage mapping relationship, the target data table corresponding to the access user identifier and the access user data size tag is searched.

[0136] In some embodiments, the storage mapping relationship includes: an association relationship between a user identifier, a user data size tag, and a stored data table.

[0137] In other embodiments, the storage mapping relationship includes: an association relationship between a user identifier, a user data size tag, a tag time and a stored data table, wherein the tag time is used to indicate a valid time of the user data size tag.

[0138] The data access request also includes a search period for the data to be accessed. In step S730, based on the pre-established storage mapping relationship, querying the target data table corresponding to the access user identifier and the access user data scale tag includes: selecting the target tag time that matches the search period from the tag time in the mapping storage relationship; and querying the target data table corresponding to the access user identifier and the access user data scale tag within the limited period of the target tag time.

[0139] The search period can be used to search for data of one or more time periods, or it can be used to search for data of one or more time points.

[0140] In step S740, the data operation instruction is executed on the data to be accessed in the target data table to obtain a data operation result.

[0141] The data access method provided in this embodiment achieves load balancing of the stored data volume among various distributed storage nodes, thereby reducing the occurrence of data skew in distributed databases and effectively avoiding the situation where data access hotspots are concentrated on a certain storage node. During the data access process, not only is the adverse impact of hotspot access concentrated on a certain storage node on database stability avoided, but also, when the user's data volume changes dynamically, the corresponding user data scale label and the corresponding group routing algorithm will be reflected in the change, thereby dynamically adapting to the change in user data volume and performing corresponding balanced storage and access, thereby achieving dynamic adaptive storage and access.

[0142] A fourth exemplary embodiment of the present disclosure provides a data storage device, which is a storage demand side including database middleware or a storage service side including database middleware.

[0143] Figure 8 The figure schematically shows a structural block diagram of a data storage device according to an embodiment of the present disclosure.

[0144] Reference Figure 8 As shown, the data storage device 800 provided by the embodiment of the present disclosure includes: a storage request receiving module 810, a storage tag determining module 820, a routing algorithm determining module 830 and a group routing module 840.

[0145] The storage request receiving module 810 is used to receive a data storage request, where the data storage request carries a requesting user identifier and data to be stored.

[0146] The storage tag determination module 820 is configured to determine a corresponding target user data size tag according to the requesting user identifier.

[0147] The routing algorithm determination module 830 is used to determine the target group routing algorithm corresponding to the target user data scale label according to the data routing group configuration information; the data routing group configuration information is used to divide user data of different scales into data tables of different storage nodes of the database, so that the data level load between each storage node is balanced.

[0148] The group routing module 840 is used to group the data to be stored and forward them to corresponding storage nodes according to the target group routing algorithm, and to establish a storage mapping relationship between the data to be stored and the stored data table.

[0149] In some embodiments, the apparatus 800 further includes: a label configuration module.

[0150] The above-mentioned label configuration module is used to: generate scale label information for the above-mentioned target user within the above-mentioned statistical period based on at least one of the stored data and database access performance of the target user within the statistical period; and generate or update the user data scale label in the above-mentioned user label configuration table based on the above-mentioned scale label information.

[0151] For other relevant details or beneficial effects of this embodiment, please refer to the contents of the first embodiment and will not be repeated here.

[0152] A fifth exemplary embodiment of the present disclosure provides a data access device.

[0153] Figure 9 The figure schematically shows a structural block diagram of a data access device according to an embodiment of the present disclosure.

[0154] Reference Figure 9 As shown, the data access device 900 provided by the embodiment of the present disclosure is a storage demand end including database middleware, including: a first access request receiving module 901, a first access tag determining module 902, a first storage table query module 903 and a forwarding module 904.

[0155] The first access request receiving module 901 is used to receive a data access request, which carries: an access user identifier and a data operation instruction for the data to be accessed; the data operation instruction includes at least one of the following: a query instruction, a modification instruction, and a deletion instruction.

[0156] The first access tag determination module 902 is configured to determine a corresponding access user data size tag according to the access user identifier.

[0157] The first storage table query module 903 is configured to query the target data table corresponding to the access user identifier and the access user data size tag according to a pre-built storage mapping relationship.

[0158] The forwarding module 904 is configured to forward data operation instructions for the data to be accessed in the target data table to the corresponding storage node.

[0159] For other relevant details or beneficial effects of this embodiment, please refer to the contents of the second embodiment and will not be repeated here.

[0160] A sixth exemplary embodiment of the present disclosure provides a device for accessing data.

[0161] Figure 10 The figure schematically shows a structural block diagram of a data access device according to another embodiment of the present disclosure.

[0162] Reference Figure 10 As shown, the data access device 1000 provided by the embodiment of the present disclosure is a storage service end including database middleware, including: a second access request receiving module 1001, a second access tag determining module 1002, a second storage table query module 1003 and a data operation module 1004.

[0163] The second access request receiving module 1001 is used to receive a data access request, which carries: an access user identifier and a data operation instruction for the data to be accessed; the data operation instruction includes at least one of the following: a query instruction, a modification instruction, and a deletion instruction.

[0164] The second access tag determination module 1002 is configured to determine a corresponding access user data size tag according to the access user identifier.

[0165] The second storage table query module 1003 is configured to query the target data table corresponding to the access user identifier and the access user data size tag according to a pre-built storage mapping relationship.

[0166] The data operation module 1004 is used to execute the data operation instruction on the data to be accessed in the target data table to obtain a data operation result.

[0167] For other relevant details or beneficial effects of this embodiment, please refer to the contents of the third embodiment and will not be repeated here.

[0168] Any number of the functional modules included in the above-mentioned apparatus 800, apparatus 900 or apparatus 1000 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. At least one of the functional modules included in the apparatus 800, apparatus 900 or apparatus 1000 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented in hardware or firmware by any other reasonable means of integrating or packaging the circuit, or can be implemented in any one of the three implementation methods of software, hardware and firmware, or in any appropriate combination of any of them. Alternatively, at least one of the functional modules included in the apparatus 800, apparatus 900 or apparatus 1000 can be at least partially implemented as a computer program module, which can perform the corresponding function when the computer program module is executed.

[0169] A seventh exemplary embodiment of the present disclosure provides an electronic device.

[0170] Figure 11 The structural block diagram of the electronic device provided by the embodiment of the present disclosure is schematically shown.

[0171] Reference Figure 11 As shown, the electronic device 1100 provided by the embodiment of the present disclosure includes a processor 1101, a communication interface 1102, a memory 1103 and a communication bus 1104, wherein the processor 1101, the communication interface 1102 and the memory 1103 communicate with each other through the communication bus 1104; the memory 1103 is used to store computer programs; the processor 1101 is used to implement the data storage method or data access method as described above when executing the program stored in the memory.

[0172] The eighth exemplary embodiment of the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the data storage method or data access method described above.

[0173] The computer-readable storage medium may be included in the device or apparatus described in the above embodiments, or may exist independently without being incorporated into the device or apparatus. The computer-readable storage medium carries one or more programs, and when the one or more programs are executed, the method according to the embodiments of the present disclosure is implemented.

[0174] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium, such as, but not limited to, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0175] It should be noted that the collection, collection, updating, analysis, processing, use, transmission, and storage of user personal information involved in the technical solutions provided by the embodiments of this disclosure all comply with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken with respect to user personal information to prevent unauthorized access to user personal information data and to maintain the security of user personal information, network security, and national security.

[0176] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0177] The foregoing description is intended only to provide specific embodiments of the present disclosure, intended to enable those skilled in the art to understand and implement the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure is not to be limited to the embodiments shown herein, but is to be construed in the broadest manner consistent with the principles and novel features claimed herein.

Claims

1. A data storage method, characterized in that: include: receiving a data storage request, the data storage request carrying a requesting user identifier and data to be stored; Determine a corresponding target user data size label according to the requesting user identifier; The method includes: performing a match in a user tag configuration table based on the requesting user identifier to obtain a matching user data scale tag; the user tag configuration table is dynamically updated over time and includes: the user identifier, the user data scale tag, and the tag time; the user data scale tag is dynamically updated according to the actual data volume; the tag time is used to indicate the valid time of the user data scale tag; and determining, based on the matching tag time corresponding to the matching user data scale tag, a target user data scale tag corresponding to the moment of receipt of the data storage request, including: comparing, based on the matching tag time and the moment of receipt, determining, among one or more matching user data scale tags, a matching user data scale tag that is valid at the moment of receipt as the target user data scale tag; Determining a target group routing algorithm corresponding to the target user data size label based on data routing group configuration information; the data routing group configuration information is used to divide user data of different sizes into data tables of different storage nodes in the database so that the data level load is balanced between the storage nodes; According to the target group routing algorithm, the data to be stored are grouped and forwarded to corresponding storage nodes, and a storage mapping relationship between the data to be stored and the stored data table is constructed.

2. The method according to claim 1, characterized in that The data routing group configuration information includes: a correspondence between a user data scale label and a group routing algorithm; The determining, according to the data routing group configuration information, a target packet routing algorithm corresponding to the target user data scale label comprises: determining, according to the corresponding relationship, a target packet routing algorithm corresponding to the target user data scale label; In the case where the target user data scale label is a user label of a first level scale, the target group routing algorithm includes: performing a first routing grouping operation on the attribute information of the data to be stored, so that the data to be stored is divided into a first number of data tables; When the target user data scale label is a user label of the second-level scale, the target group routing algorithm includes: performing a first routing grouping operation on the attribute information of the data to be stored to obtain a first logical grouping result; performing a second routing grouping operation on the first logical grouping result, so that the data to be stored is divided into a second number of data tables; wherein the second-level scale is larger than the first-level scale, and the second number is larger than the first number.

3. The method according to claim 2, characterized in that The first routing grouping operation includes: a hash modulo operation; The second routing grouping operation includes: performing grouping operation according to a preset time period; The attribute information includes: requesting user identification.

4. The method according to claim 1, wherein The tag time is represented by the starting time of the user data scale tag, and is used to indicate that the time period from the previous starting time to the next starting time is valid; The method of comparing the matching tag time with the receiving time and determining the matching user data scale tag that is valid at the receiving time among the one or more matching user data scale tags as the target user data scale tag includes: If the matching tag time includes multiple matching start times and all of them are earlier than the receiving time, determining the matching user data scale tag corresponding to the latest matching start time among the multiple matching start times as the target user data scale tag; When the matching tag time includes multiple matching start times and the receiving time falls within a period between a first matching start time and a second matching start time, the matching user data scale tag corresponding to the first matching start time is determined as the target user data scale tag.

5. The method according to claim 1, wherein The tag time is represented by the duration information of the user data scale tag; The method of comparing the matching tag time with the receiving time and determining the matching user data scale tag that is valid at the receiving time among the one or more matching user data scale tags as the target user data scale tag includes: When the matching tag time includes one or more matching duration information, the target matching duration information within which the receiving moment falls is determined; and the matching user data size tag corresponding to the target matching duration information is determined as the target user data size tag.

6. The method according to claim 1, characterized in that Also includes: Generating scale label information for the target user within the statistical period based on at least one of the stored data and database access performance of the target user within the statistical period; Generate or update the user data scale label in the user label configuration table according to the scale label information.

7. The method according to any one of claims 1 to 6, characterized in that The method is applied to a storage service end including a database middleware or a storage demand end including a database middleware; The method includes grouping the data to be stored and forwarding it to the corresponding storage node according to the target group routing algorithm, and constructing a storage mapping relationship between the data to be stored and the stored data table, including: Executing the target group routing algorithm on the attribute information of the data to be stored, and dividing the data to be stored into corresponding data tables of different storage nodes of the database; Build an association between the data to be stored and the stored data table; According to the requesting user identifier, target user data scale label and corresponding label time of the data to be stored, the association relationship is associated and expanded to obtain a multi-dimensional storage mapping relationship; the storage mapping relationship includes: the association relationship between the user identifier, user data scale label, label time and the stored data table.

8. A data access method, characterized in that: Applied to a storage demand side including database middleware, the method includes: Receive a data access request, the data access request carrying: an access user identifier and a data operation instruction for the data to be accessed; the data operation instruction includes at least one of the following: a query instruction, a modification instruction, and a deletion instruction; Determining a corresponding access user data scale label based on the access user identifier; the method comprising: performing a match in a user label configuration table based on the access user identifier to obtain a matching user data scale label; the user label configuration table is dynamically updated over time and includes: a user identifier, a user data scale label, and a label time; the user data scale label is dynamically updated with the actual data volume; the label time is used to indicate the valid time of the user data scale label; determining, based on the matching label time corresponding to the matching user data scale label, the access user data scale label corresponding to the moment of receipt of the data access request, the method comprising: comparing, based on the matching label time and the moment of receipt, determining, among one or more matching user data scale labels, a matching user data scale label that is valid at the moment of receipt as the access user data scale label; According to the pre-built storage mapping relationship, query the target data table corresponding to the access user identifier and the access user data scale label; The data operation instructions for the data to be accessed in the target data table are forwarded to the corresponding storage node.

9. A data access method, characterized in that: Applied to a storage service client including database middleware, the method includes: Receive a data access request, the data access request carrying: an access user identifier and a data operation instruction for the data to be accessed; the data operation instruction includes at least one of the following: a query instruction, a modification instruction, and a deletion instruction; Determining a corresponding access user data scale label based on the access user identifier; the method comprising: performing a match in a user label configuration table based on the access user identifier to obtain a matching user data scale label; the user label configuration table is dynamically updated over time and includes: a user identifier, a user data scale label, and a label time; the user data scale label is dynamically updated with the actual data volume; the label time is used to indicate the valid time of the user data scale label; determining, based on the matching label time corresponding to the matching user data scale label, the access user data scale label corresponding to the moment of receipt of the data access request, the method comprising: comparing, based on the matching label time and the moment of receipt, determining, among one or more matching user data scale labels, a matching user data scale label that is valid at the moment of receipt as the access user data scale label; According to the pre-built storage mapping relationship, query the target data table corresponding to the access user identifier and the access user data scale label; The data operation instruction is executed on the data to be accessed in the target data table to obtain a data operation result.

10. The method according to claim 8 or 9, characterized in that The data access request also carries: a search period for the data to be accessed; The storage mapping relationship includes: the association relationship between the user identifier, the user data scale label, the label time and the stored data table; the label time is used to indicate the valid time of the user data scale label; The step of querying the target data table corresponding to the access user identifier and the access user data scale tag according to the pre-built storage mapping relationship includes: Filtering the target tag time that matches the search period from the tag time storing the mapping relationship; Within a limited period of the target tag time, query the target data table corresponding to the access user identifier and the access user data scale tag.

11. A data storage device, characterized in that: The device is a storage demand end including database middleware or a storage service end including database middleware, and the device includes: A storage request receiving module, configured to receive a data storage request, wherein the data storage request carries a requesting user identifier and data to be stored; A storage tag determination module is configured to determine a corresponding target user data scale tag based on the requesting user identifier. This module includes: performing a match in a user tag configuration table based on the requesting user identifier to obtain a matching user data scale tag; the user tag configuration table is dynamically updated over time and includes: a user identifier, a user data scale tag, and a tag time; the user data scale tag is dynamically updated with the actual data volume; the tag time indicates the validity period of the user data scale tag; and determining a target user data scale tag corresponding to the moment of receipt of the data storage request based on the matching tag time corresponding to the matching user data scale tag, including: comparing the matching tag time with the moment of receipt and determining, among one or more matching user data scale tags, a matching user data scale tag that is valid at the moment of receipt as the target user data scale tag. a routing algorithm determination module, configured to determine a target group routing algorithm corresponding to the target user data size label based on data routing group configuration information; the data routing group configuration information is configured to divide user data of different sizes into data tables of different storage nodes in the database, so that the data level load is balanced between the storage nodes; The group routing module is used to group the data to be stored and forward them to corresponding storage nodes according to the target group routing algorithm, and to establish a storage mapping relationship between the data to be stored and the stored data table.

12. A data access device, characterized in that: The device is a storage demand end including database middleware, and the device includes: A first access request receiving module is configured to receive a data access request, wherein the data access request carries: an access user identifier and a data operation instruction for the data to be accessed; the data operation instruction includes at least one of the following: a query instruction, a modification instruction, and a deletion instruction; A first access tag determination module is configured to determine a corresponding access user data scale tag based on the access user identifier, including: performing a match in a user tag configuration table based on the access user identifier to obtain a matching user data scale tag; the user tag configuration table is dynamically updated over time and includes: a user identifier, a user data scale tag, and a tag time; the user data scale tag is dynamically updated with the actual data volume; the tag time is used to indicate the valid time of the user data scale tag; and determining, based on the matching tag time corresponding to the matching user data scale tag, the access user data scale tag corresponding to the moment of receipt of the data access request, including: comparing the matching tag time with the moment of receipt, and determining, among one or more matching user data scale tags, a matching user data scale tag that is valid at the moment of receipt as the access user data scale tag; A first storage table query module is used to query the target data table corresponding to the access user identifier and the access user data scale label according to a pre-built storage mapping relationship; The forwarding module is used to forward the data operation instruction for the data to be accessed in the target data table to the corresponding storage node.

13. A data access device, characterized in that: The device is a storage service end including a database middleware, and the device includes: A second access request receiving module is configured to receive a data access request, wherein the data access request carries: an access user identifier and a data operation instruction for the data to be accessed; the data operation instruction includes at least one of the following: a query instruction, a modification instruction, and a deletion instruction; A second access tag determination module is configured to determine a corresponding access user data scale tag based on the access user identifier, including: performing a match in a user tag configuration table based on the access user identifier to obtain a matching user data scale tag; the user tag configuration table is dynamically updated over time and includes: a user identifier, a user data scale tag, and a tag time; the user data scale tag is dynamically updated with the actual data volume; the tag time is used to indicate the valid time of the user data scale tag; and determining, based on the matching tag time corresponding to the matching user data scale tag, the access user data scale tag corresponding to the moment of receipt of the data access request, including: comparing the matching tag time with the moment of receipt, and determining, among one or more matching user data scale tags, a matching user data scale tag that is valid at the moment of receipt as the access user data scale tag. A second storage table query module is used to query the target data table corresponding to the access user identifier and the access user data scale label according to a pre-built storage mapping relationship; The data operation module is used to execute the data operation instruction on the data to be accessed in the target data table to obtain a data operation result.

14. An electronic device, characterized in that: The processor, the communication interface, the memory and the communication bus are connected to each other via the communication bus. Memory for storing computer programs; A processor, configured to implement the method according to any one of claims 1 to 10 when executing a program stored in a memory.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

Images