Data access method, device, electronic device, medium and program product

Abstract

The present disclosure relates to a data access method, device, electronic device, medium, and program product. The data access method includes: determining whether a first application corresponding to target data to be accessed is a hot application, wherein the hot application is an application whose data volume is greater than a preset threshold; if the first application is a hot application, accessing the target data in an independent index, wherein any index shard used for the independent index only includes data of one application, and the access includes querying and / or writing. This data access method can access data according to the application dimension, thereby improving the data reading and writing efficiency in the case of massive data.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a data access method, device, electronic device, medium, and program product. Background Art

[0002] With the development of the internet, both the amount of data stored and the amount of data added are growing exponentially, leading to efficiency issues with current data access. For example, when developers register apps on an application platform and need to access private information such as user profile pictures, they must authorize the user. Once the user authorizes, the platform's backend stores the authorization relationship between the user and the app, known as an authorization pair. Currently, the number of authorization pairs (userId and appId) is in the billions, and as the number of users and applications grows, the number of authorization pairs will increase rapidly. With this massive amount of data, efficiency in data access needs to be further considered. Summary of the Invention

[0003] The present disclosure provides a data access method, apparatus, electronic device, medium, and program product to at least solve the above-mentioned problems in the related technologies, and may not solve any of the above-mentioned problems.

[0004] According to a first aspect of an embodiment of the present disclosure, a data access method is provided, which includes: determining whether a first application corresponding to target data to be accessed is a hot application, wherein the hot application is an application whose data volume is greater than a preset threshold; in a case where the first application is a hot application, accessing the target data in an independent index, wherein any index shard used for the independent index only includes data of one application, and the access includes querying and / or writing.

[0005] Optionally, the method further includes: when the first application is not a hot application, accessing the target data in a public index, wherein any index shard used for the public index includes data of multiple applications.

[0006] Optionally, accessing the target data in the public index includes: determining a first index shard for the public index based on identification information of the first application; and accessing the target data in the first index shard.

[0007] Optionally, the index shards used for the public index include a preset number of index shards, the identification information includes an application identification number, and determining the first index shard for the public index based on the identification information of the first application includes: performing a modulo operation on the application identification number of the first application and the preset number; and determining the first index shard based on the result of the modulo operation.

[0008] Optionally, it also includes: when the amount of data of the second application in the second index shard for the public index grows to exceed the preset threshold, creating a third index shard for the independent index for the second application; synchronizing the data of the second application from the second index shard to the third index shard, and accessing the data of the second application in the third index shard.

[0009] Optionally, synchronizing the data of the second application from the second index shard to the third index shard includes: writing the new data of the second application in the second index shard and the third index shard together, and synchronizing the first historical data of the second application in the second index shard to the third index shard.

[0010] Optionally, after synchronizing the first historical data of the second application in the second index shard to the third index shard, it also includes: performing data comparison on the first historical data and the second historical data of the second application in the third index shard; and determining that the synchronization is completed when the first historical data and the second historical data are consistent based on the data comparison result.

[0011] Optionally, accessing the data of the second application in the third index shard further includes: deleting the data of the second application in the second index shard.

[0012] According to a second aspect of an embodiment of the present disclosure, a data access device is provided, comprising: an application determination unit configured to determine whether a first application corresponding to target data to be accessed is a hot application, wherein the hot application is an application whose data volume is greater than a preset threshold; a first access unit configured to access the target data in an independent index when the first application is a hot application, wherein any index shard used for the independent index includes data of only one application, and the access includes querying and / or writing.

[0013] Optionally, it also includes: a second access unit, configured to access the target data in a public index when the first application is not a hot application, wherein any index shard used for the public index includes data of multiple applications.

[0014] Optionally, the second access unit is configured to: determine a first index shard for the public index based on identification information of the first application; and access the target data in the first index shard.

[0015] Optionally, the index shards used for the public index include a preset number of index shards, the identification information includes an application identification number, and the second access unit is configured to: perform a modulo operation on the application identification number of the first application and the preset number; and determine the first index shard based on the result of the modulo operation.

[0016] Optionally, it also includes: a shard switching unit, configured to create a third index shard for the independent index for the second application when the amount of data of the second application in the second index shard for the public index grows to exceed the preset threshold; synchronize the data of the second application from the second index shard to the third index shard, and access the data of the second application in the third index shard.

[0017] Optionally, the shard switching unit is configured to: write the new data of the second application in the second index shard and the third index shard together, and synchronize the first historical data of the second application in the second index shard to the third index shard.

[0018] Optionally, it also includes: a data comparison unit, configured to compare the first historical data with the second historical data of the second application in the third index shard; when it is determined based on the data comparison result that the first historical data and the second historical data are consistent, the synchronization is determined to be completed.

[0019] Optionally, the shard switching unit is further configured to: delete the data of the second application in the second index shard.

[0020] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement the data access method according to the present disclosure.

[0021] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor is prompted to execute the data access method according to the present disclosure as described above.

[0022] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, comprising computer instructions, which implement the data access method according to the present disclosure when executed by a processor.

[0023] The technical solutions provided by the embodiments of the present disclosure bring at least the following beneficial effects:

[0024] According to the data access method, device, electronic device, medium and program product disclosed in the present invention, data can be accessed according to the application dimension, thereby improving the data reading and writing efficiency in the case of massive data, and can isolate the data of hot applications with large data volumes into separate index shards without affecting online business, further improving the data reading and writing efficiency of each application.

[0025] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute an improper limitation of the present disclosure.

[0027] Figure 1 is a flowchart illustrating a data access method according to an exemplary embodiment of the present disclosure;

[0028] Figure 2 is a flow chart illustrating an optional data access method according to an exemplary embodiment of the present disclosure;

[0029] Figure 3 is a schematic diagram illustrating index sharding according to an exemplary embodiment of the present disclosure;

[0030] Figure 4 is a demonstration diagram illustrating an index shard switching process according to an exemplary embodiment of the present disclosure;

[0031] Figure 5 is a block diagram illustrating a data access device according to an exemplary embodiment of the present disclosure;

[0032] Figure 6 is a block diagram illustrating an electronic device 600 according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0033] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0034] It should be noted that the terms "first," "second," and the like in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the numbers used in this manner are interchangeable where appropriate so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The implementation methods described in the following examples do not represent all implementation methods consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with certain aspects of the present disclosure as detailed in the appended claims.

[0035] It should be noted that the phrase "at least one of the items" in this disclosure includes three types of parallel situations: "any one of the items", "a combination of any multiple items of the items", and "all of the items". For example, "including at least one of A and B" includes the following three parallel situations: (1) including A; (2) including B; (3) including A and B. For another example, "performing at least one of step 1 and step 2" includes the following three parallel situations: (1) performing step 1; (2) performing step 2; and (3) performing steps 1 and 2.

[0036] The present disclosure provides a data access method, device, electronic device, medium and program product, which can access data according to the application dimension, improve the data reading and writing efficiency in the case of massive data, and can isolate the data of hot applications with large data volumes into separate index shards without affecting online business, further improving the data reading and writing efficiency of each application.

[0037] Refer to the following Figures 1 to 6 The data access method, apparatus, electronic device, medium, and program product according to exemplary embodiments of the present disclosure are described in detail.

[0038] Figure 1 is a flowchart illustrating a data access method according to an exemplary embodiment of the present disclosure.

[0039] Reference Figure 1 In step S101, it can be determined whether the first application corresponding to the target data to be accessed is a hot application. Here, the hot application can be an application whose data volume is greater than a preset threshold. Furthermore, the method of measuring the data volume can be, but is not limited to, measuring based on the storage size of the data or the number of entries of a specific type of data. Those skilled in the art can set it according to actual conditions; and the preset threshold of the data volume can also be set by those skilled in the art according to actual conditions, and this disclosure does not limit it here.

[0040] Next, in step S102, when the first application is a hot application, the target data can be accessed in an independent index. Here, any index shard used for an independent index only includes the data of one application, and access includes querying and / or writing. In other words, any index shard used for an independent index only includes the data of one hot application, so the number of index shards used for an independent index is determined by the number of hot applications. By isolating the data of hot applications with large data volumes into separate index shards, the data reading and writing efficiency of both hot applications and non-hot applications can be improved.

[0041] Figure 2 is a flowchart illustrating an alternative data access method according to an exemplary embodiment of the present disclosure.

[0042] Reference Figure 2 In step S203, when the first application is not a hot application, the target data can be accessed in the public index. Here, any index shard used for the public index includes data of multiple applications. In other words, any index shard used for the public index can include data of multiple non-hot applications, and the number of index shards used for the public index can be set by those skilled in the art according to actual conditions. By storing data of multiple non-hot applications with small data volumes in the same index shard, unnecessary resources can be avoided without reducing data reading and writing efficiency.

[0043] According to an exemplary embodiment of the present disclosure, the first index shard for the public index can be determined based on the identification information of the first application; then, the target data can be accessed in the first index shard. Here, the identification information can be used to identify the application, and then the identification information can be used to distinguish different applications. Further, those skilled in the art can set the identification information according to actual conditions, for example, the identification information can be set to the application identification number (appId), but the present disclosure is not limited to this. First, the index shard in the public index is determined by the identification information, and then the data is accessed in the corresponding index shard, which can improve the efficiency of data access and reduce the probability of errors during data access. As an example, the index shard for the public index includes a preset number of index shards, and the identification information includes an application identification number. On this basis, the application identification number of the first application can be modulo-divided by the preset number; then, the first index shard can be determined based on the result of the modulo-division operation. Routing the index shards through the modulo-division operation can improve the data reading and writing efficiency of non-hotspot applications.

[0044] According to an exemplary embodiment of the present disclosure, if the amount of data for a second application in a second index shard used for a public index exceeds a preset threshold, a third index shard can be created for the second application for an independent index. The second application's data is synchronized from the second index shard to the third index shard, and the second application's data can be accessed from the third index shard. By setting up separate index shards for applications with large data volumes, data read and write efficiency for these applications can be improved. Since the switching of index shards is a process, data access may still be required during this process. To avoid impacting online services, the second application's new data can be written to both the second and third index shards, and the second application's first historical data in the second index shard can be synchronized to the third index shard. By writing the new data to both the independent index and the public index, data synchronization is ensured to ensure efficiency and avoid omissions. Furthermore, data synchronization methods include the following: 1) reading data directly from the public index and writing it to the independent index; 2) synchronizing data from the MySQL binlog and writing it to the independent index; 3) synchronizing data from a Hive table to Kafka and then writing it to the independent index. The above three synchronization methods are only examples and are not limited in this disclosure.

[0045] According to an exemplary embodiment of the present disclosure, after synchronizing the first historical data of the second application in the second index shard to the third index shard, a data comparison may be performed between the first historical data and the second historical data of the second application in the third index shard; then, if the first historical data and the second historical data are determined to be consistent based on the data comparison result, the synchronization is determined to be complete. Here, the data comparison may be a sampling comparison or a comprehensive comparison, and those skilled in the art may configure it based on actual circumstances. Through data comparison, data consistency after data synchronization can be ensured, thereby improving the reliability of index shard switching.

[0046] According to an exemplary embodiment of the present disclosure, after the index shard is switched, since the data of the second application is accessed in the third index shard, that is, the data of the second application is no longer accessed in the second index shard, the data of the second application in the second index shard can also be deleted to save resources.

[0047] In order to facilitate understanding of the above content, the following takes the authorization data described in the background technology as an example, combined with Figure 3 and Figure 4 Provide a detailed description. Figure 3 is a schematic diagram illustrating index sharding according to an exemplary embodiment of the present disclosure, Figure 4 is a demonstration diagram illustrating an index shard switching process according to an exemplary embodiment of the present disclosure.

[0048] For the authorization pair data described above, MySQL can currently be used to index and shard the data based on the user identification number (userId) dimension to perform queries in separate libraries and tables. However, when accessing authorization pair data based on the application identification number (appId) dimension, MySQL can no longer meet the usage requirements. Therefore, Elasticsearch (ES) can be used to index the authorization pair data based on the appId dimension. However, if only one index is used in ES to store such a large amount of data, there will be efficiency issues. In addition, due to the existence of hot apps, that is, a single app has nearly 100 million levels of data, hot apps may affect the data reading and writing efficiency of other apps, so index sharding is required based on appId. In addition, when multiple index shards are created based on the appId dimension, there is a situation where a non-hot app in a certain index shard develops into a hot app as the business grows, which will also affect the data reading and writing efficiency of each application.

[0049] Therefore, refer to Figure 3 , the index can be divided into two parts: public index and independent index. As an example, the public index can be further divided into 10 index shards, namely index shards index1, index2, ..., index10. Then, when accessing data, the data is routed to the corresponding index shard based on the appId corresponding to the data modulo (mod) 10; the data of each index shard of the independent index is a single hot app. Apps with a data volume greater than 50 million can be defined as hot apps, such as Figure 3 As shown, the independent index can be divided into index shards indexA and indexB. In addition, data query and write operations can be performed through a pre-configured shard module. This shard module then determines whether the app is a hot app. If it is a hot app, data query and write operations are performed in the independent index based on the index shard configuration. If it is not a hot app, data query and write operations are performed in the public index based on the modulo result of the appId. As an example, this can be achieved in the above shard module through the following configuration style:

[0050]

[0051]

[0052] Here, infra_oauth_authorize represents the index alias, which is used to distinguish the index of other business data. readLargeIndexKey represents a query (i.e., read) operation, writeLargeIndexKey represents a write (i.e., write) operation, doubleWriteIndexKey represents a double write operation, and shardNum represents a modulo operation. 22669 represents the appId of a popular app, 10000 represents the appId of a non-popular app, and 10 represents the number of index shards used for the public index.

[0053] Reference Figure 4 For example, when the data volume of the app with appId 10000 in the public index exceeds the threshold of 50 million, the following series of configurations can be used to switch the data of the app from the public index to the independent index. Figure 4 As shown, in phase 1, dual index writing is possible. Specifically, the new data of the app can be written to the corresponding index shards of the independent index and the public index at the same time based on the following configuration:

[0054]

[0055] Then, in phase 2, the app's historical data can be synchronized from the Hive table to the independent index. The synchronized data can then be compared. Specifically, data comparison can be achieved through the following configuration:

[0056]

[0057] Here, compareGrayKey is used to determine which data to compare. Its format can be defined as: modulo base; effective portion; whitelist; blacklist. For this custom format, the effective portion supports numeric ranges and dot separators, while the whitelist and blacklist only support dot separators. Furthermore, the effective portion can contain only dots or be left blank, and the whitelist and blacklist can also be left blank, for example, in the formats 100;;; and 100;1;;. When executing the configuration, the matching order for the various components in this format can be: first match the blacklist, then modulo match the effective portion, and finally match the whitelist. Specifically, in the configuration "100;0-50;384728,483894;", 100 represents the modulo base, 0-50 represents the effective portion, 384728,483894 represents the whitelist, and the blacklist is left blank. Using this compareGrayKey, sampled data can be compared, avoiding inefficiencies when the data volume is too large.

[0058] Then, in phase 3, after data synchronization is complete and data consistency is guaranteed, you can add the app's appId, 10000, to readLargeIndexKey to switch subsequent read operations of the app's data to an independent index. This can be achieved through the following configuration:

[0059]

[0060]

[0061] Then, in stage 4, the dual write configuration can be removed so that subsequent write operations for the app data are performed only in the independent index. This can be achieved through the following configuration:

[0062]

[0063] Finally, in stage 5, the data of the app with appId 10000 can be deleted in the public index based on the reIndex instruction. Figure 4 The configurations of stages 1 to 5 shown can switch the app's data from the public index to the independent index. During subsequent data reading and writing, corresponding data access is performed based on the independent index. This automatically isolates the data of hot applications with large data volumes into separate index shards without affecting online business.

[0064] Therefore, according to the data access method disclosed in the present invention, data can be accessed according to the application dimension, which improves the data reading and writing efficiency in the case of massive data, and can isolate the data of hot applications with large data volumes into separate index shards without affecting online business, further improving the data reading and writing efficiency of each application.

[0065] Figure 5 is a block diagram showing a data access device. Figure 5 The data access device 500 includes: an application determining unit 501 and a first access unit 502 .

[0066] The application determining unit 501 may determine whether the first application corresponding to the target data to be accessed is a hot application. Here, a hot application is an application whose data volume is greater than a preset threshold.

[0067] The first access unit 502 may access the target data in the independent index when the first application is a hot application. Here, any index shard used for the independent index only includes data of one application, and the access may include querying and / or writing.

[0068] According to an exemplary embodiment of the present disclosure, the data access device 500 may further include a second access unit (not shown). The second access unit may access the target data in the public index when the first application is not a hot application. Here, any index shard used for the public index includes data of multiple applications.

[0069] According to an exemplary embodiment of the present disclosure, the second access unit may determine a first index shard for a public index based on identification information of the first application; and then, may access target data in the first index shard.

[0070] According to an exemplary embodiment of the present disclosure, the index shards for a public index may include a preset number of index shards, the identification information may include an application identification number, and the second access unit may perform a modulo operation on the application identification number of the first application and the preset number; then, the first index shard may be determined based on the result of the modulo operation.

[0071] According to an exemplary embodiment of the present disclosure, the data access device 500 may further include a shard switching unit (not shown). When the amount of data of the second application in the second index shard for the common index increases to exceed a preset threshold, the shard switching unit may create a third index shard for an independent index for the second application; synchronize the data of the second application from the second index shard to the third index shard; and access the data of the second application in the third index shard.

[0072] According to an exemplary embodiment of the present disclosure, the shard switching unit may write the new data of the second application in both the second index shard and the third index shard, and synchronize the first historical data of the second application in the second index shard to the third index shard.

[0073] According to an exemplary embodiment of the present disclosure, the data access device 500 may further include a data comparison unit (not shown). The data comparison unit may compare the first historical data with the second historical data of the second application in the third index shard; and then, if it is determined based on the data comparison result that the first historical data and the second historical data are consistent, the synchronization is determined to be complete.

[0074] According to an exemplary embodiment of the present disclosure, the shard switching unit may further delete the data of the second application in the second index shard.

[0075] According to the data access method, device, electronic device, medium and program product disclosed in the present invention, data can be accessed according to the application dimension, thereby improving the data reading and writing efficiency in the case of massive data, and can isolate the data of hot applications with large data volumes into separate index shards without affecting online business, further improving the data reading and writing efficiency of each application.

[0076] According to an embodiment of the present disclosure, an electronic device may be provided. Figure 6 It is a block diagram of an electronic device 600 according to an embodiment of the present disclosure, which includes at least one memory 601 and at least one processor 602. The at least one memory stores a set of computer-executable instructions. When the computer-executable instruction set is executed by the at least one processor, the data access method according to the embodiment of the present disclosure is executed.

[0077] As an example, the electronic device 600 may be a PC, a tablet device, a personal digital assistant, a smart phone, or other device capable of executing the above-mentioned instruction set. Here, the electronic device 600 is not necessarily a single electronic device, but may also be any device or circuit collection capable of executing the above-mentioned instructions (or instruction set) individually or in combination. The electronic device 600 may also be part of an integrated control system or system manager, or may be configured as a portable electronic device interconnected with a local or remote (e.g., via wireless transmission) interface.

[0078] In electronic device 600, processor 602 may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller, or a microprocessor. By way of example and not limitation, processor 602 may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0079] The processor 602 can execute instructions or codes stored in the memory 601, wherein the memory 601 can also store data. Instructions and data can also be sent and received over the network via the network interface device, wherein the network interface device can use any known transmission protocol.

[0080] The memory 601 may be integrated with the processor 602, for example, by placing RAM or flash memory within an integrated circuit microprocessor or the like. Furthermore, the memory 601 may comprise a separate device, such as an external disk drive, a storage array, or any other storage device usable by a database system. The memory 601 and the processor 602 may be operatively coupled or may communicate with each other, for example, via an I / O port, a network connection, or the like, such that the processor 602 can access files stored in the memory 601.

[0081] In addition, the electronic device 600 may further include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, a mouse, a touch input device, etc.) All components of the electronic device may be connected to each other via a bus and / or a network.

[0082] According to an embodiment of the present disclosure, a computer-readable storage medium may also be provided, wherein when the instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor is prompted to execute the data access method of the embodiment of the present disclosure. Examples of computer-readable storage media here include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), card storage (such as, multimedia card, secure digital (SD) card or ultra-fast digital (XD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk and any other device, any other device configured to store the computer program and any associated data, data files and data structures in a non-transitory manner and provide the computer program and any associated data, data files and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the above-mentioned computer-readable storage medium can be run in an environment deployed in a computer device such as a client, a host, an agent device, a server, etc. In addition, in one example, the computer program and any associated data, data files and data structures are distributed on a networked computer system so that the computer program and any associated data, data files and data structures are stored, accessed and executed in a distributed manner by one or more processors or computers.

[0083] According to an embodiment of the present disclosure, a computer program product is provided, including computer instructions, which implement the data access method of the embodiment of the present disclosure when the computer instructions are executed by a processor.

[0084] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0085] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A data access method, characterized in that: The data access method includes: Determining whether a first application corresponding to the target data to be accessed is a hot application, wherein the hot application is an application whose data volume is greater than a preset threshold; In a case where the first application is a hot application, accessing the target data in an independent index, wherein any index shard used for the independent index includes data of only one application, and the access includes querying and / or writing; In a case where the first application is not a hot application, the target data is accessed in a public index, wherein any index shard used for the public index includes data of multiple applications.

2. The data access method according to claim 1, wherein: The accessing of the target data in the public index includes: Determining a first index shard for the public index based on the identification information of the first application; The target data is accessed in the first index shard.

3. The data access method according to claim 2, wherein: The index shards used for the public index include a preset number of index shards, the identification information includes an application identification number, and determining a first index shard used for the public index based on the identification information of the first application includes: performing a modulo division operation on the application identification number of the first application and the preset number; Based on a result of the modulo operation, the first index fragment is determined.

4. The data access method according to claim 1, wherein: Also includes: When the data volume of the second application in the second index shard for the public index increases to exceed the preset threshold, creating a third index shard for the independent index for the second application; The data of the second application is synchronized from the second index shard to the third index shard, and the data of the second application is accessed in the third index shard.

5. The data access method according to claim 4, wherein: The synchronizing the data of the second application from the second index shard to the third index shard includes: The newly added data of the second application is written into both the second index shard and the third index shard, and the first historical data of the second application in the second index shard is synchronized into the third index shard.

6. The data access method according to claim 5, wherein: After synchronizing the first historical data of the second application in the second index shard to the third index shard, the method further includes: Performing a data comparison on the first historical data and the second historical data of the second application in the third index shard; If it is determined based on the data comparison result that the first historical data and the second historical data are consistent, it is determined that the synchronization is completed.

7. The data access method according to claim 6, wherein: The accessing the data of the second application in the third index shard further includes: Delete the data of the second application in the second index shard.

8. A data access device, characterized in that: The data access device comprises: an application determining unit configured to determine whether a first application corresponding to the target data to be accessed is a hot application, wherein the hot application is an application whose data volume is greater than a preset threshold; a first access unit configured to, when the first application is a hot application, access the target data in an independent index, wherein any index shard used for the independent index includes data of only one application, and the access includes querying and / or writing; The second access unit is configured to access the target data in a public index when the first application is not a hot application, wherein any index shard used for the public index includes data of multiple applications.

9. The data access device according to claim 8, wherein: The second access unit is configured to: Determining a first index shard for the public index based on the identification information of the first application; The target data is accessed in the first index shard.

10. The data access device according to claim 9, wherein: The index shards used for the public index include a preset number of index shards, the identification information includes an application identification number, and the second access unit is configured to: performing a modulo division operation on the application identification number of the first application and the preset number; Based on a result of the modulo operation, the first index fragment is determined.

11. The data access device according to claim 8, wherein: Also includes: A shard switching unit is configured to create a third index shard for the independent index for the second application when the data amount of the second application in the second index shard for the common index increases to exceed the preset threshold; The data of the second application is synchronized from the second index shard to the third index shard, and the data of the second application is accessed in the third index shard.

12. The data access device according to claim 11, wherein: The shard switching unit is configured to: The newly added data of the second application is written into both the second index shard and the third index shard, and the first historical data of the second application in the second index shard is synchronized into the third index shard.

13. The data access device according to claim 12, wherein: Also includes: a data comparison unit, configured to compare the first historical data with the second historical data of the second application in the third index shard; If it is determined based on the data comparison result that the first historical data and the second historical data are consistent, it is determined that the synchronization is completed.

14. The data access device according to claim 13, wherein: The shard switching unit is further configured to: Delete the data of the second application in the second index shard.

15. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the data access method according to any one of claims 1 to 7.

16. A computer-readable storage medium, characterized in that When the instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor is prompted to perform the data access method according to any one of claims 1 to 7.

17. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by a processor, the data access method according to any one of claims 1 to 7 is implemented.

Images