Data acquisition methods and related products

CN118012914BActive Publication Date: 2026-08-14TENCENT MUSIC ENTERTAINMENT TECH (SHENZHEN) CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-02-21
Publication Date
2026-08-14

AI Technical Summary

Technical Problem

[0003]然而现有方式中,应用程序的线程所请求的数据不在缓存中时,分布式缓存系统需要针对每个请求都做一遍,从DB中查询出所需数据返回给应用程序的操作,又因DB部署在磁盘上而对数据的处理速度慢,这导致了业务突发高峰期时,分布式缓存系统对各请求的响应效率被大大降低

Benefits of technology

[0033]面对多个数据获取线程预获得同一数据的请求,本申请实施例将多个数据获取线程分配给同一目标服务器,及选择其中一线程作为数据查询线程从数据源中加载目标数据到目标服务器的内存,可避免不同服务器的多个同需求线程去重复查询数据源造成资源损耗。此外,促使目标服务器的内存将目标数据共享给同属于该目标服务器的数据获取线程,可减少(如同一服务器的)多个线程重复查询分布式缓存的次数和耗时,提高大量请求并发获取数据时的响应效率。

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118012914B_ABST
    Figure CN118012914B_ABST
Patent Text Reader

Abstract

This application discloses a data acquisition method and related products. The method includes: when faced with multiple data acquisition threads requesting the same data, if the target data is not in the distributed cache, assigning at least two data acquisition threads to the same target server; and running a data query thread on the target server to find the target data from the data source and write it to the target server's memory. Assigning multiple data acquisition threads to the same target server, and selecting one thread to load the target data from the data source into the target server's memory, avoids resource consumption caused by multiple threads with the same request on different servers repeatedly querying the data source. Furthermore, enabling the target server's memory to share the target data with data acquisition threads belonging to the same target server reduces the number of times and time spent repeatedly querying the distributed cache by multiple threads, improving response efficiency when a large number of requests are concurrently acquiring data.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of Internet technology, and in particular to data acquisition methods and related products. Background Technology

[0002] In high-concurrency environments, a large number of external requests flood the data source (such as a database) to retrieve data, which can easily cause the database to crash due to excessive load. To solve this problem, distributed caching mechanisms have emerged. In short, a distributed cache is a system that shares data among different servers. If the data needed by the application is already distributed on a cache shared by the server cluster, it no longer needs to access the database or other data sources. The application can simply search for the data in the cache and return it to the application, thus greatly reducing the I / O load on the database.

[0003] However, in the existing approach, when the data requested by the application's thread is not in the cache, the distributed caching system needs to perform the operation of querying the required data from the database and returning it to the application for each request. Since the database is deployed on the disk, the data processing speed is slow. This results in a significant reduction in the response efficiency of the distributed caching system to each request during peak business periods.

[0004] Therefore, there is an urgent need to provide an effective solution. Summary of the Invention

[0005] This application provides a data acquisition method and related products to improve the response efficiency of a distributed caching system to concurrent data acquisition requests.

[0006] The first aspect of this application provides a data acquisition method, including:

[0007] If multiple clients initiate the same data retrieval thread to the distributed cache within a preset time period, and the target data requested by the data retrieval thread is not in the distributed cache, at least two of the data retrieval threads will be assigned to the same target server.

[0008] Select one of the at least two data acquisition threads allocated to the target server as a data query thread, run the data query thread to search for the target data in the data source, and write the searched target data into the memory of the target server;

[0009] The target data stored in the memory of the target server is shared with the other data acquisition threads on the target server, excluding the data query thread.

[0010] Optionally, selecting one of the at least two data acquisition threads allocated from the target server as a data query thread includes:

[0011] The system drives at least two data acquisition threads allocated to the target server to compete for the resource lock, and the first data acquisition thread to acquire the resource lock is designated as the data query thread.

[0012] Optionally, selecting one of the at least two data acquisition threads allocated from the target server as a data query thread includes:

[0013] The levels of each data acquisition thread assigned to the target server are compared, and the data acquisition thread with the highest level is selected as the data query thread.

[0014] Optionally, sharing the target data stored in the memory of the target server with other data acquisition threads on the target server besides the data query thread includes:

[0015] The target server's memory is configured to be in data-sharing mode;

[0016] The target data stored in the memory of the target server is simultaneously shared with other data acquisition threads on the target server, excluding the data query thread.

[0017] Optionally, sharing the target data stored in the memory of the target server with other data acquisition threads on the target server besides the data query thread includes:

[0018] The target server's memory is configured to be in data-sharing mode;

[0019] Based on the ranking comparison results among the data acquisition threads in the target server, the target data stored in memory is shared sequentially with the other data acquisition threads in the target server.

[0020] Optionally, after writing the retrieved target data into the memory of the target server, the method further includes:

[0021] Write the target data stored in the memory of the target server into the distributed cache;

[0022] The system notifies some of the other data acquisition threads allocated to the target server to retrieve the target data from the memory, while the other threads retrieve the target data from the distributed cache.

[0023] Optionally, after writing the retrieved target data into the memory of the target server, the method further includes:

[0024] The target data stored in the memory of the target server is written into the distributed cache, which is composed of the target server and other servers besides the target server;

[0025] If the other servers are also allocated the data acquisition thread, instruct the data acquisition thread on the other servers to retrieve the target data from the distributed cache.

[0026] A second aspect of this application provides an electronic device, including:

[0027] Central processing unit, memory, and input / output interfaces;

[0028] The memory is either a short-term storage memory or a persistent storage memory;

[0029] The central processing unit is configured to communicate with the memory and execute instructions in the memory to perform the method described in the first aspect of the embodiments of this application or any specific implementation thereof.

[0030] A third aspect of this application provides a computer-readable storage medium including instructions that, when executed on a computer, cause the computer to perform the method described in the first aspect of this application or any specific implementation thereof.

[0031] A fourth aspect of this application provides a computer program product comprising instructions or a computer program, which, when run on a computer, causes the computer to perform the method described in the first aspect of this application or any specific implementation thereof.

[0032] As can be seen from the above technical solutions, the embodiments of this application have at least the following advantages:

[0033] When faced with multiple data acquisition threads requesting the same data, this embodiment of the application allocates multiple data acquisition threads to the same target server, and selects one thread as the data query thread to load the target data from the data source into the target server's memory. This avoids resource consumption caused by multiple threads with the same request on different servers repeatedly querying the data source. Furthermore, by enabling the target server's memory to share the target data with data acquisition threads belonging to the same target server, the number of times and time spent repeatedly querying the distributed cache by multiple threads (on the same server) can be reduced, improving response efficiency when a large number of requests are concurrently acquiring data. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings.

[0035] It should be noted that although the steps in the flowcharts (if any) involved in the embodiments are drawn sequentially according to the arrows, unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts involved in the embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps.

[0036] Figure 1 This is a schematic diagram of a system architecture for the data acquisition method in an embodiment of this application;

[0037] Figure 2 This is a flowchart illustrating a data acquisition method according to an embodiment of this application;

[0038] Figure 3 This is another flowchart illustrating the data acquisition method according to an embodiment of this application;

[0039] Figure 4 This is a schematic diagram of the existing traditional queuing process using the "single-piece mode double-lock loading" method.

[0040] Figure 5 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0042] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0043] In the following description, expressions such as "one specific implementation" or "one specific example" describe a subset of all possible embodiments. However, it is understood that "one specific implementation" or "one specific example" can be the same or different subset of all possible embodiments and can be combined with each other without conflict. In the following description, the term "multiple" means at least two. When a certain value mentioned in this application reaches a threshold (if it exists), in some specific examples, it may include the former being greater than the latter. When "any" or "at least one" or similar expressions are mentioned, it specifically refers to any one of the listed examples or any combination of these examples.

[0044] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0045] Please see Figure 1 , Figure 1 This is a schematic diagram of the system architecture provided in an embodiment of this application. For example... Figure 1As shown, the system architecture may include a business server 100 and a terminal cluster. The terminal cluster may include terminal devices 200a, 200b, 200c, ..., 200n, etc., which are terminal devices (i.e., clients). The business server 100 may be an independent physical server, a server cluster composed of multiple physical servers, or a distributed system. It may also be a cloud server providing basic cloud computing services such as cloud databases, cloud services, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. Terminal devices may be smartphones, tablets, laptops, desktop computers, PDAs, mobile internet devices (MIDs), wearable devices (such as smartwatches and smart bracelets), smart computers, smart vehicles, and other smart terminals. The business server 100 can establish communication connections with each terminal device in the terminal cluster, and the terminal devices in the terminal cluster can also establish communication connections with each other. The above-mentioned communication connection is not limited to a specific method. It can be connected directly or indirectly through wired communication or wireless communication, etc. The specific method can be determined according to the actual application scenario, and this application does not impose any restrictions.

[0046] When multiple terminal devices in a terminal cluster require the same target data, these terminal devices can each initiate a data retrieval thread (referred to as a task or thread) to the distributed cache to pre-obtain the target data. The business server 100 can then allocate at least two threads to the same target server. The target server can then control one of its allocated threads to search for the target data in a data source (such as a database) and write the retrieved data into the target server's memory. This allows the memory to directly share the stored target data with other threads within the target server, reducing the number of queries and the time spent by multiple threads on the same server accessing the database or distributed cache.

[0047] It should be noted that the methods provided in the embodiments of this application can be implemented jointly by the terminal device and the server as described above, or they can be implemented entirely on the server side, or they can be implemented entirely on the terminal device side. The specific implementation can be determined according to the actual application scenario, and no restrictions are imposed here.

[0048] The method described in this application will be further explained in detail below.

[0049] Please see Figure 2 The first aspect of this application provides a specific embodiment of a data acquisition method, which includes the following operation steps:

[0050] Step 21: Assign at least two concurrent threads that retrieve the same data to the same target server.

[0051] If multiple clients initiate the same data retrieval thread to the distributed cache within a preset time period, and the target data requested by the data retrieval thread is not in the distributed cache, at least two data retrieval threads will be assigned to the same target server. Here, "at least two data retrieval threads" can specifically refer to some or all of the threads that are requesting the same target data.

[0052] In practical applications, distributed caching is a storage medium that allows data to be manipulated over a network. It typically consists of a cluster of multiple servers (or nodes), with cached data evenly distributed across the nodes. Distributed caching is characterized by higher read / write speeds than databases and other disk storage methods; however, data is not suitable for long-term storage. The aforementioned preset time period can be a period with a zero time interval, encompassing situations where multiple threads initiate simultaneously. This refers to multiple tasks needing to be processed concurrently (concurrency). The aforementioned multiple data acquisition threads can refer to multiple tasks occurring or executing simultaneously within the same process (the lifespan of a computer program from startup to shutdown is called a process). Each task can be called a thread; a process is analogous to a processing workshop, and threads are analogous to individual employees, each capable of working simultaneously in the workshop.

[0053] Step 22: Run a data acquisition thread on the target server to load the target data from the data source into the target server's memory.

[0054] The inventors have discovered that, in comparison, memory is a data storage medium privately controlled by a process (which can be considered a local cache). It can be directly read and written without network interaction, and its read and write speed is much higher than that of distributed cache. The aforementioned data source can be a database (DB) or a database server. A DB is a storage medium that can operate on data over a network. Its characteristics are large data storage capacity and support for various query modes. However, because DBs are deployed on disks, and disks have slow data processing speeds, DBs are slower than caches in terms of data processing speed. Furthermore, DBs are clearly not on the same order of magnitude slower than memory. Therefore, the inventors propose that when multiple threads concurrently request the same target data (i.e., initiate the same thread), one of the at least two data acquisition threads allocated by the target server can be selected as the data query thread. The data query thread runs to search for the target data in the data source (such as a DB) and writes the found target data into the target server's memory, thereby reducing the number of DB queries and the time spent on data feedback.

[0055] Step 23: Share the target data stored in the target server's memory with other data acquisition threads on the target server.

[0056] Specifically, the target data stored in the target server's memory is shared with other data acquisition threads on the target server, excluding the data query thread.

[0057] In summary, when faced with multiple data acquisition threads requesting the same data, this embodiment assigns these threads to the same target server and selects one thread as the data query thread to load the target data from the data source into the target server's memory. This avoids resource consumption caused by multiple threads with the same request on different servers repeatedly querying the data source. Furthermore, by enabling the target server's memory to share the target data with data acquisition threads belonging to the same target server, the number of times and time spent repeatedly querying the distributed cache by multiple threads (on the same server) can be reduced, improving response efficiency when a large number of requests are concurrently acquiring data.

[0058] Based on the examples above, some specific possible implementation examples will be provided below. In practical applications, the implementation content of these examples can be combined or implemented separately as needed according to the corresponding functional principles and application logic. If combined, the execution order between the combined examples can be determined according to their respective processing logic, which can be determined by the actual scenario.

[0059] based on Figure 2 As an example implementation, step 22 may include: driving at least two data acquisition threads allocated to the target server to compete for the resource lock, and using the first data acquisition thread that acquires the resource lock as the data query thread.

[0060] The resource lock acquisition mentioned here can be understood as a queuing process for attempting to acquire a lock: multiple tasks need to be processed simultaneously, but they all need to compete for control of the same resource. In this case, multiple tasks can attempt to lock the same resource, and the task that successfully acquires the lock will gain control of the current resource. For example, a data acquisition thread that successfully acquires the lock will have the opportunity to act as a data query thread to search for the target data in the database. That is, compared to tasks that fail to acquire the lock, tasks that successfully acquire the lock have priority in obtaining the target data.

[0061] based on Figure 2 As an example, another possible implementation of step 22 may include: comparing the levels of the data acquisition threads allocated to the target server, and designating the data acquisition thread with the highest level as the data query thread. For example, this level could be a time-series level based on the thread's occurrence time, or a level between the clients initiating the thread, such as a distinction between Level 1 VIP users, Level 2 VIP users, or non-VIP users; there are no specific limitations.

[0062] based on Figure 2In some specific examples, step 23 may include (simultaneous wake-up mechanism): setting the target server's memory to data sharing mode; and simultaneously sharing the target data stored in the target server's memory with other data acquisition threads on the target server, excluding the data query thread.

[0063] like Figure 3 As shown, three clients, A, B, and C, initiate data acquisition threads A, B, and C simultaneously within a preset time period. All three threads request promotional activity information (i.e., target data). Therefore, if no promotional activity information is found in the cache, threads A, B, and C can be assigned to the same server for processing. For example, if the three threads enter the locked area to compete for control of the lock, when thread A acquires control (i.e., the resource lock), threads B and C will enter a waiting-to-wake-up (i.e., waiting-to-notify) mode. This wake-up can be used to notify threads B and C to simultaneously acquire the target data. Thread A can then check the cache again for promotional activity information (this step can be skipped, i.e., A directly queries the database). If not found, thread A queries the database for existing promotional activity information and stores it in the server's memory. Afterward, thread A can simultaneously wake up threads B and C and share the data in memory with threads B and C, allowing the three threads to quickly proceed with subsequent processes, such as analyzing the obtained promotional activity information.

[0064] Considering that simultaneous sharing (i.e., simultaneous data retrieval) may put pressure on memory, therefore based on Figure 2 In some specific examples, step 23 may include (a priority wake-up mechanism based on rank): setting the target server's memory to a data-sharing mode; and sharing the target data stored in memory with other data acquisition threads on the target server according to the ranking comparison results among the data acquisition threads on the target server. This priority wake-up mechanism can be seen as a way of queuing for data acquisition; specifically, it can be about notifying other data acquisition threads allocated to the target server to acquire the target data from memory in sequence.

[0065] The inventors have discovered that the aforementioned simultaneous wake-up mechanism has more advantages than the priority wake-up mechanism based on rank settings. These advantages include avoiding the need for threads B and C to queue again to obtain the target data, and allowing threads B and C in the simultaneous wake-up mechanism to synchronously and directly share the target data contributed by memory. In short, the simultaneous wake-up mechanism reduces the need for multiple queuing-based concurrent access control among multiple threads, thus improving system response efficiency. Furthermore, with... Figure 4Taking the traditional queuing method based on "single-piece mode double-lock loading" (i.e., multiple queues for locking and unlocking) as an example, after thread A queues and successfully acquires control (i.e., successfully acquires the lock), threads B and C, which failed to acquire the lock, need to wait for the lock to be released. After thread A queries the cache again and finds no data (this step can be omitted), thread A queries the database and stores the target data found in the database into the memory of the target server. Thread A writes the data in memory into the cache, releases the lock, and executes the subsequent process. Afterwards, thread B, which failed to acquire the lock, queues again (thread C becomes the one that failed to acquire the lock again) and acquires the lock released by A. It then uses the acquired lock to query the cache again, and can find the target data in the cache. After thread B retrieves the target data and releases the lock, thread C acquires the lock and will repeatedly query the cache for the target data, and so on.

[0066] As can be seen, the traditional queuing method based on "single-piece mode double-lock loading" involves multiple tasks running simultaneously, but only one task can acquire resource control (i.e., the lock). Other tasks must queue and wait for the task with control to unlock and release the resource before they are allowed to compete for resource control again (i.e., they are eligible to queue for the lock again). Furthermore, the inventors have found that the "single-piece mode double-lock loading" scheme suffers significant performance overhead in distributed caching scenarios. Therefore, compared to the traditional queuing method based on "single-piece mode double-lock loading," the simultaneous wake-up mechanism provided in this application can reduce the queuing process for locking and unlocking (i.e., avoid threads other than A from queuing). Simultaneously, by using shared data memory, it reduces the number of cache queries and the pressure on the cache. That is, once one thread stores the target data in the memory of the same server, other threads can synchronously and quickly share the memory data without queuing (because memory access is faster than cache).

[0067] Considering the limited memory resources of servers or certain practical needs, in practice, it is advisable to distribute the memory pressure on the target server. For example, based on... Figure 1 In some specific examples, after step 22, the method of this application embodiment may further include: writing the target data stored in the memory of the target server into a distributed cache; notifying some threads in the other data acquisition threads allocated to the target server to acquire the target data from memory, and other threads to acquire the target data from the distributed cache.

[0068] Considering that there are multiple servers constituting a distributed caching system, threads requesting the same target data can be distributed to other servers in the server cluster for processing, and the memory of one server cannot be shared by other servers, in order to improve the overall response efficiency when concurrently requesting the same target data, based on Figure 1In some specific examples, after step 22, the method of this application embodiment may further include: writing the target data stored in the memory of the target server into a distributed cache, the distributed cache being composed of the target server and other servers besides the target server; if other servers are also allocated data acquisition threads, instructing the data acquisition threads on the other servers to acquire the target data from the distributed cache.

[0069] In summary, the simultaneous wake-up mechanism provided in this application uses a batch sleep-wake-up scheme to ensure that the same data does not need to be loaded multiple times in the same process, nor does it need to be queued multiple times for access control. This ensures that multiple data requests at the same time only query the DB once, and the application of shared memory can reduce the number of queries to the cache and the time consumption.

[0070] Compared to Figure 2 The examples shown illustrate that the additional or refined examples or possible implementation methods mentioned above may not necessarily be executed in actual implementation. If more than two examples or possible implementation methods are added, these examples or possible implementation methods can be implemented in combination or individually. If implemented in combination, the execution order between the combined examples can be determined according to their respective processing logic, depending on the actual scenario.

[0071] Please see Figure 5 The electronic device 500 in this application embodiment may include one or more central processing units (CPUs) 501 and a memory 505, wherein the memory 505 stores one or more applications or data.

[0072] The memory 505 can be volatile or persistent storage. The program stored in the memory 505 can include one or more modules, each module including a series of instruction operations on the electronic device. Furthermore, the central processing unit 501 can be configured to communicate with the memory 505 and execute the series of instruction operations stored in the memory 505 on the electronic device 500.

[0073] Electronic device 500 may also include one or more power supplies 502, one or more wired or wireless network interfaces 503, one or more input / output interfaces 504, and / or one or more operating systems, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0074] The central processing unit 501 can perform the operations performed by the first aspect or any specific method embodiment of the first aspect, which will not be described in detail here.

[0075] This application provides a computer-readable storage medium including instructions that, when executed on a computer, cause the computer to perform the method as described in the first aspect or any specific implementation thereof.

[0076] This application provides a computer program product containing instructions or computer programs, which, when run on a computer, causes the computer to perform the method described in the first aspect or any specific implementation thereof.

[0077] It is understood that, in the various embodiments of this application, the sequence number of each step does not imply the order of execution. The execution order of each step should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The operations added or refined to the above-described methods, systems, or devices (if any) may not necessarily be executed in specific implementations. If more than two operations are added, these operations can be implemented in combination or individually, depending on the actual scenario.

[0078] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the system (if it exists) and device described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0079] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system or apparatus, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0080] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0081] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0082] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product (computer program product) is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a business server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A data acquisition method, characterized in that, include: If multiple clients initiate the same data retrieval thread to the distributed cache within a preset time period, and the target data requested by the data retrieval thread is not in the distributed cache, at least two of the data retrieval threads will be assigned to the same target server. Select one of the at least two data acquisition threads allocated to the target server as a data query thread, run the data query thread to search for the target data in the data source, and write the searched target data into the memory of the target server; The target data stored in the memory of the target server is shared with the other data acquisition threads on the target server, excluding the data query thread.

2. The method according to claim 1, characterized in that, Selecting one of the at least two data acquisition threads allocated from the target server as a data query thread includes: The system drives at least two data acquisition threads allocated to the target server to compete for the resource lock, and the first data acquisition thread to acquire the resource lock is designated as the data query thread.

3. The method according to claim 1, characterized in that, Selecting one of the at least two data acquisition threads allocated from the target server as a data query thread includes: The levels of each data acquisition thread assigned to the target server are compared, and the data acquisition thread with the highest level is selected as the data query thread.

4. The method according to any one of claims 1 to 3, characterized in that, The step of sharing the target data stored in the memory of the target server with other data acquisition threads on the target server besides the data query thread includes: The target server's memory is configured to be in data-sharing mode; The target data stored in the memory of the target server is simultaneously shared with other data acquisition threads on the target server, excluding the data query thread.

5. The method according to any one of claims 1 to 3, characterized in that, The step of sharing the target data stored in the memory of the target server with other data acquisition threads on the target server besides the data query thread includes: The target server's memory is configured to be in data-sharing mode; Based on the ranking comparison results among the data acquisition threads in the target server, the target data stored in memory is shared sequentially with the other data acquisition threads in the target server.

6. The method according to any one of claims 1 to 3, characterized in that, After writing the retrieved target data into the memory of the target server, the method further includes: Write the target data stored in the memory of the target server into the distributed cache; The system notifies some of the other data acquisition threads allocated to the target server to retrieve the target data from the memory, while the other threads retrieve the target data from the distributed cache.

7. The method according to any one of claims 1 to 3, characterized in that, After writing the retrieved target data into the memory of the target server, the method further includes: The target data stored in the memory of the target server is written into the distributed cache, which is composed of the target server and other servers besides the target server; If the other servers are also allocated the data acquisition thread, instruct the data acquisition thread on the other servers to retrieve the target data from the distributed cache.

8. An electronic device, characterized in that, include: Central processing unit, memory, and input / output interfaces; The memory is either a short-term storage memory or a persistent storage memory; The central processing unit is configured to communicate with the memory and execute instructions in the memory to perform the method according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, Includes instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1 to 7.

10. A computer program product comprising instructions or a computer program, characterized in that, When the computer program product is run on a computer, it causes the computer to perform the method as described in any one of claims 1 to 7.

Citation Information

Patent Citations

  • Method, device and equipment for acquiring hardware information of distributed storage server

    CN108173715A

  • Timing task execution method, distributed server cluster and electronic device

    CN109582466A