Distributed cache key value pair architecture data reading method and device, equipment and medium
By using plug-ins on the cache node to read the data target range, the problem that the terminal needs to process all data is solved, bandwidth usage is reduced, and the performance and user experience of the distributed cache architecture are improved.
Patent Information
- Application Number
- CN202410977409.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-19
- Publication Date
- 2024-11-22
AI Technical Summary
In a distributed cache architecture, when users only need to extract part of the data found by the cache node, the terminal needs to perform additional data processing, resulting in heavy tasks and increasing data transmission bandwidth usage, affecting performance.
Use plug-ins on the cache node to read the data found through key-value pairs in the target range, directly returning the actual data content required by the user, reducing the terminal's data processing burden and unnecessary bandwidth usage.
It reduces the data processing burden of the terminal, reduces unnecessary data bandwidth usage, improves the performance of the distributed cache architecture, and thus improves the user experience.
Smart Images

Figure CN119003025A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of information processing technology, and in particular to a distributed cache key-value pair architecture data reading method, device, equipment and medium. Background Art
[0002] Currently, with the widespread application of distributed cache, user data is usually sliced and stored in cache nodes in the form of key-value pairs.
[0003] In the related technology, generally speaking, the user initiates a data request to the cache node through the client, and the cache node finds the complete value and returns it according to the matching rules of the key-value pair. However, in actual applications, it is found that users often only want to extract part of the data in the complete value found (that is, they do not need to extract all the data). In this case, the client needs to process the data returned by the cache node and read the data actually needed by the user, which requires the terminal to undertake additional data processing tasks, resulting in a heavier task for the terminal. In addition, it also increases the unnecessary occupation of the data transmission bandwidth between the terminal and the cache node, affecting the performance of the distributed cache architecture.
[0004] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the invention
[0005] The embodiments of the present application provide a distributed cache key-value pair architecture data reading method, device, equipment and medium, which can reduce the data processing burden of the terminal, reduce unnecessary data bandwidth occupancy, improve the performance of the distributed cache architecture, and thus improve the user experience.
[0006] On the one hand, an embodiment of the present application provides a distributed cache key-value pair architecture data reading method, which is applied to a cache node and includes the following steps:
[0007] Receiving a data request sent by a terminal; wherein the data request includes a data processing requirement and a key matching the data requested by the user;
[0008] According to the key, determine the value corresponding to the key as the data to be processed;
[0009] According to the data processing requirements, determine the plug-in that matches the data processing requirements;
[0010] The target data is read from the data to be processed through the plug-in; wherein the target data is the data within the target range of the data to be processed;
[0011] Send the target data to the terminal.
[0012] Optionally, before receiving the data request sent by the terminal, the method further includes:
[0013] Receiving registration plug-in information sent by a registration interface on the terminal; wherein the registration plug-in information includes all plug-ins that have been registered through the registration interface on the terminal;
[0014] According to the registered plug-in information, register and preload all plug-ins that have been registered through the registration interface on the terminal.
[0015] Optionally, the plug-in includes a cropping sub-plug-in, a splicing sub-plug-in, a compression sub-plug-in, and a custom sub-plug-in.
[0016] Optionally, according to the key, determining a value corresponding to the key as data to be processed includes:
[0017] Retrieve all key-value pair information cached in the cache node;
[0018] When key-value pair information related to the key is matched in all key-value pair information, the value in the key-value pair information is read as the to-be-processed data corresponding to the key.
[0019] Optionally, the method further comprises:
[0020] When no key-value pair information related to the key is matched in all key-value pair information, the key is sent to the underlying storage system so that the underlying storage system can search for the corresponding value based on the key;
[0021] Receives the value sent by the underlying storage system as data to be processed.
[0022] Optionally, after receiving the value sent by the underlying storage system as the data to be processed, the method further includes:
[0023] Combine the key and the value sent by the underlying storage system into a set of key-value pair information;
[0024] Cache a set of key-value pairs in the cache node.
[0025] Optionally, the underlying storage system is a Ceph distributed storage system.
[0026] On the other hand, an embodiment of the present application provides a distributed cache key-value pair architecture data reading method, which is applied to a terminal, including:
[0027] generating a data request in response to a triggered data request instruction;
[0028] Sending a data request to a cache node; wherein the data request includes a data processing requirement and a key matching the data requested by the user;
[0029] Receive target data returned by the cache node; wherein the target data is read by the cache node from the data to be processed through the plug-in; the plug-in is determined by the cache node according to the data processing requirements; the data to be processed is determined by the cache node according to the key.
[0030] Optionally, before generating a data request in response to a data request instruction triggered by a user, the method further includes:
[0031] In response to a plug-in registration instruction triggered by a user, register the plug-in in the registration interface;
[0032] Based on all the plug-ins that have been registered in the registration interface, the registered plug-in information is generated and sent to the cache node.
[0033] Optionally, the plug-in includes a cropping sub-plug-in, a splicing sub-plug-in, a compression sub-plug-in, and a custom sub-plug-in.
[0034] Optionally, the method further comprises:
[0035] In response to the triggered plug-in update instruction, the plug-in is updated.
[0036] On the other hand, an embodiment of the present application provides a distributed cache key-value pair architecture data reading device, the device comprising:
[0037] A request receiving unit, configured to receive a data request sent by a terminal; wherein the data request includes a data processing requirement and a key matching the data requested by the user;
[0038] A data determination unit, used to determine, according to the key, a value corresponding to the key as data to be processed;
[0039] A plug-in determination unit, used to determine a plug-in matching the data processing requirement according to the data processing requirement;
[0040] A data reading unit, used to read target data from the data to be processed through a plug-in; wherein the target data is data within a target range in the data to be processed;
[0041] The data sending unit is used to send target data to the terminal.
[0042] On the other hand, an embodiment of the present application provides a distributed cache key-value pair architecture data reading device, the device comprising:
[0043] A request generating unit, configured to generate a data request in response to a triggered data request instruction;
[0044] A request sending unit, used to send a data request to a cache node; wherein the data request includes a data processing requirement and a key matching the data requested by the user;
[0045] A data receiving unit is used to receive target data returned by a cache node; wherein the target data is read by the cache node from the data to be processed through a plug-in; the plug-in is determined by the cache node according to data processing requirements; the data to be processed is determined by the cache node according to a key;
[0046] On the other hand, an embodiment of the present application provides an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the above-mentioned distributed cache key-value pair architecture data reading method when executing the computer program.
[0047] On the other hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned distributed cache key-value pair architecture data reading method is implemented.
[0048] The embodiment of the present application utilizes a plug-in loaded at a cache node to read the data obtained through a key-value pair search according to a target range, thereby being able to directly return the actual data content required by the user, which can reduce the data processing burden of the terminal, reduce unnecessary data bandwidth usage, and improve the performance of the distributed cache architecture, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] Figure 1 It is a structural diagram of a distributed cache key-value pair architecture provided by related technologies;
[0050] Figure 2 It is a schematic diagram of an implementation environment of a distributed cache key-value pair architecture data reading method provided in an embodiment of the present application;
[0051] Figure 3 It is a flow chart of a distributed cache key-value pair architecture data reading method provided in an embodiment of the present application;
[0052] Figure 4 It is a structural diagram of a distributed cache key-value pair architecture provided by an embodiment of the present application;
[0053] Figure 5 It is a flowchart of another distributed cache key-value pair architecture data reading method provided in an embodiment of the present application;
[0054] Figure 6 It is a structural diagram of a distributed cache key-value pair architecture registration plug-in provided in an embodiment of the present application;
[0055] Figure 7 It is a structural diagram of another distributed cache key-value pair architecture provided in an embodiment of the present application;
[0056] Figure 8It is a structural diagram of a distributed cache key-value pair architecture data reading device provided in an embodiment of the present application;
[0057] Fig. 9 It is a structural diagram of another distributed cache key-value pair architecture data reading device provided in an embodiment of the present application;
[0058] Fig.10 It is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0059] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are only examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the attached claims.
[0060] It is understood that the terms "first", "second", etc. used in this application can be used to describe various concepts in this article, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another concept. For example, without departing from the scope of the embodiment of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination".
[0061] The terms "at least one", "multiple", "each", "any", etc. used in this application, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.
[0062] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.
[0063] Currently, with the widespread application of distributed cache, user data is usually sliced and stored in cache nodes in the form of key-value pairs.
[0064] like Figure 1 As shown, Figure 1It is a structural diagram of a distributed cache key-value pair architecture provided by the related technology. In the related technology, it is usually the terminal (such as the client. For the convenience of description, the following embodiments will use the client to describe) to split the user's data into slices (such as Figure 1 Several segments of slices shown in ), thereby obtaining multiple groups of key-value pair information and storing them in the cache node and the underlying storage system. When the user initiates a data request to the cache node through the client, the cache node finds the complete value and returns it according to the matching rules of the key-value pair. However, it is found in actual applications that when the user only wants to extract part of the data in the complete value found (that is, without extracting all the data), for example, the user's data is split and sliced according to the granularity of 4MB, and written and stored in the cache node, but the user usually does not need to read the complete data when reading the data. For example, the user needs to read the range of 1MB in the 4MB fragment data. At this time, the client needs to process the obtained 4MB fragment data, which makes the task undertaken by the client heavier. For different usage scenarios, the client needs customized adaptation. In addition, because the cache node needs to return the complete data to the client, the bandwidth between the client and the cache node is increased and unnecessary, which affects the performance of the distributed cache architecture and the user experience.
[0065] In view of this, the embodiments of the present application provide a distributed cache key-value pair architecture data reading method, device, equipment and medium, which utilizes a plug-in loaded at the cache node to read the data obtained through key-value pair search according to the target range, thereby being able to directly return the actual data content required by the user, thereby reducing the data processing burden of the terminal, reducing unnecessary data bandwidth occupancy, and improving the performance of the distributed cache architecture, thereby improving the user experience.
[0066] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.
[0067] The specific implementation of the embodiment of the present application is described in detail below in conjunction with the accompanying drawings. First, a distributed cache key-value pair architecture data reading method provided in the embodiment of the present application is described in conjunction with the accompanying drawings.
[0068] Please refer to Figure 2 , Figure 2 The schematic diagram of the implementation environment of a distributed cache key-value pair architecture data reading method provided in the embodiment of the present application is shown. In the implementation environment, the main subjects involved include a distributed cache key-value pair architecture data reading device 110 and a server 120.
[0069] Specifically, the distributed cache key-value pair architecture data reading device 110 may be installed with a processor, and the processor is communicatively connected to the server 120 .
[0070] Server 120 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), as well as big data and artificial intelligence platforms.
[0071] The processor and the server 120 may establish a communication connection through a wireless network. The wireless network uses standard communication technology and / or protocols, and the network may be set to the Internet or any other network, such as but not limited to a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, or any combination of a wireless network, a private network, or a virtual private network. Moreover, the above-mentioned software and hardware entities may use the same communication connection mode or different communication connection modes, and this application does not impose any specific restrictions on this.
[0072] Of course, it is understandable that Figure 2 The implementation environment in the example is only some optional application scenarios of the distributed cache key-value pair architecture data reading method provided in the embodiment of the present application, and the actual application is not fixed. Figure 2 The environment shown is not specifically limited in this application.
[0073] like Figure 3 As shown, Figure 3 This is a flow chart of a distributed cache key-value pair architecture data reading method provided by an embodiment of the present application, the method comprising the following steps:
[0074] S100, receiving a data request sent by a terminal; wherein the data request includes a data processing requirement and a key matching the data requested by the user.
[0075] In the embodiment of the present application, the execution subject may be a cache node in a distributed cache architecture, the terminal (exemplarily, the terminal may be a client) may be a mobile communication device such as a smart phone or a tablet computer, and the data may be unstructured data, such as files, pictures, and videos, etc. Exemplarily, when a user initiates a data request to read any picture through a smart phone, the mobile phone as the client will send the data request to the cache node.
[0076] Furthermore, the data request includes data processing requirements and keys that match the data requested by the user. It is understandable that in a distributed cache architecture, data can be stored in a key-value pair format. When data is written and stored in a cache node, each key uniquely identifies a value, which can store data simply and efficiently, and store the results of recent accesses through key-value pairs, reducing the access pressure on the backend database. In addition, data processing requirements can include different types of requirements such as video compression and image cropping.
[0077] S200: According to the key, determine a value corresponding to the key as data to be processed.
[0078] In an embodiment of the present application, after the cache node receives a data request sent by the client, it determines the value corresponding to the key based on the correspondence between key-value pairs in the cache node according to the key carried in the data request that matches the data requested by the user, and uses it as the data to be processed.
[0079] Specifically, in a possible implementation, according to the key, determining a value corresponding to the key as data to be processed includes:
[0080] Retrieve all key-value pair information cached in the cache node;
[0081] When key-value pair information related to the key is matched in all key-value pair information, the value in the key-value pair information is read as the to-be-processed data corresponding to the key.
[0082] In an embodiment of the present application, all key-value pair information cached in the cache node can be retrieved. When it is determined that key-value pair information related to the key is matched among all key-value pair information, that is, when the key as the identifier is consistent, the value in the key-value pair information is read as the data to be processed corresponding to the key.
[0083] Please refer to Figure 4 , Figure 4This is a structural diagram of a distributed cache key-value pair architecture provided in an embodiment of the present application. Multiple sets of key-value pair information are stored in the cache node. When the cache node receives a data request sent by the client, it can retrieve whether there is any key-value pair information matching it based on the key carried in the data request, and when it is determined that key-value pair information related to the key is matched among all key-value pair information, the value in the key-value pair information is read as the data to be processed corresponding to the key.
[0084] Specifically, in a possible implementation manner, the method further includes:
[0085] When no key-value pair information related to the key is matched in all key-value pair information, the key is sent to the underlying storage system so that the underlying storage system can search for the corresponding value based on the key;
[0086] Receives the value sent by the underlying storage system as data to be processed.
[0087] In the example of the present application, there may be a situation where the key-value pair information related to the key cannot be matched in all the key-value pair information cached in the cache node. In this case, the key can be sent to the underlying storage system.
[0088] like Figure 4 As shown, when the underlying storage system receives the key sent by the cache node, it can search for the corresponding value based on the key and return the searched value to the cache node.
[0089] Furthermore, when the cache node receives a value sent by the underlying storage system, it can use the value as data to be processed.
[0090] As an optional embodiment, after receiving the value sent by the underlying storage system as the data to be processed, the method further includes:
[0091] Combine the key and the value sent by the underlying storage system into a set of key-value pair information;
[0092] Cache a set of key-value pairs in the cache node.
[0093] In an embodiment of the present application, after receiving the value sent by the underlying storage system, the key and the value can be combined into a set of key-value pair information, and the set of key-value pair information can be cached in the cache node. If a subsequent user initiates the same data request again, the cache node can directly retrieve the set of key-value pair information and quickly locate the data to be processed without accessing the underlying storage system again, thereby improving the data reading speed and reducing the access pressure on the database in the underlying storage system.
[0094] As an optional embodiment, the underlying storage system is a Ceph distributed storage system.
[0095] In an embodiment of the present application, when building a distributed cache key-value pair architecture, the Ceph distributed storage system can be selected as the underlying storage system. As a high-performance, distributed storage system, Ceph can provide a high-availability, scalable storage environment, and dynamically distribute data through the CRUSH algorithm, thereby ensuring data consistency and redundancy and enhancing the fault tolerance of the Ceph distributed storage system.
[0096] S300: Determine, according to the data processing requirement, a plug-in that matches the data processing requirement.
[0097] In the embodiment of the present application, after receiving the data request, the cache node can also determine a plug-in that matches the data processing requirement according to the data processing requirement carried in the data request. A plug-in refers to an application program written in accordance with certain interface specifications.
[0098] Specifically, in a possible implementation manner, before receiving the data request sent by the client, the method further includes:
[0099] Receiving registration plug-in information sent by the registration interface on the client; wherein the registration plug-in information includes all plug-ins that have been registered by the registration interface on the client;
[0100] According to the registered plug-in information, all the plug-ins registered through the registration interface on the client are registered and preloaded.
[0101] In the embodiment of the present application, there is also an initialization phase of the distributed cache key-value pair architecture, that is, before receiving the data request sent by the client, receiving the registration plug-in information sent by the registration interface on the client. The registration plug-in information includes all plug-ins that have been registered with the registration interface on the client.
[0102] It is understandable that the user may actively select different types of plug-ins or customize the plug-ins by the user, and register the plug-ins in the registration interface on the client.
[0103] Furthermore, after receiving the registered plug-in information, the cache node can register and preload all plug-ins registered with the registration interface on the client according to the registered plug-in information, thereby completing the initialization of the cache node, so that the cache node can subsequently call the plug-in for data processing.
[0104] As an optional embodiment, the plug-in includes a cropping sub-plug-in, a splicing sub-plug-in, a compression sub-plug-in, and a custom sub-plug-in.
[0105] It is understandable that the plug-in can be divided into different plug-ins such as a cropping sub-plug-in, a splicing sub-plug-in, a compression sub-plug-in, and a custom sub-plug-in according to different data processing functions.
[0106] Among them, the cropping sub-plug-in can crop the acquired data to be processed according to the specified range, so as to return the target data actually needed by the user; the splicing sub-plug-in can splice multiple segments of acquired data to be processed, or it can first crop them through the cropping sub-plug-in and then splice the obtained cropped segments to obtain the target data actually needed by the user; the compression sub-plug-in can compress the acquired data to be processed according to the specified compression algorithm, so as to obtain the compressed target data; the custom sub-plug-in can be customized by the user according to his own usage needs or actual scenario usage needs in advance. The data processing logic is customized into a custom sub-plug-in, so as to perform custom data processing on the acquired data to be processed, and finally obtain the target data that meets the actual needs of the user.
[0107] S400. Reading target data from the data to be processed through the plug-in; wherein the target data is data within a target range in the data to be processed.
[0108] In the present application example, after the data to be processed and the plug-in to be called are determined, the plug-in can be called to process the data to be processed, thereby obtaining the final target data that meets the actual needs of the user. The target data can be part of the data read from the data to be processed according to the user's target range.
[0109] It is understandable that the data to be processed can be combined and processed by calling multiple plug-ins, for example, the data to be processed is first cropped, and then the cropped data is compressed to obtain the final target data.
[0110] In the embodiment of the present application, the storage decoupling of the plug-in and the distributed cache key-value pair architecture can effectively improve the user's convenience of use, and different types of plug-ins can be used to improve the scalability of use.
[0111] S500: Send the target data to the terminal.
[0112] In the embodiment of the present application, after the data processing is completed to obtain the final target data, the target data can be sent to the client so that the client can provide the target data to the user.
[0113] It is understandable that in the process of distributed caching key-value pair architecture target data, the client only needs to receive the user's data request and forward it to the cache node, and receive the target data returned by the cache node. Therefore, compared with the client that also needs to process data in the existing related technology, the distributed cache key-value pair architecture data reading method provided by this application can effectively reduce the data processing burden of the client. In addition, since the cache node processes the data to be processed to obtain the target data and then sends it to the client, it can effectively reduce unnecessary data bandwidth occupation, improve the performance of the distributed cache architecture, and thus improve the user experience.
[0114] like Figure 5 As shown, Figure 5 1 is a flow chart of another distributed cache key-value pair architecture data reading method provided in an embodiment of the present application, the method comprising the following steps:
[0115] S600, generating a data request in response to a triggered data request instruction;
[0116] S700, sending the data request to a cache node; wherein the data request includes a data processing requirement and a key matching the data requested by the user;
[0117] S800, receiving target data returned by the cache node; wherein the target data is read by the cache node from the data to be processed through a plug-in; the plug-in is determined by the cache node according to the data processing requirements; the data to be processed is determined by the cache node according to the key.
[0118] In an embodiment of the present application, the execution subject may be a client in a distributed cache key-value pair architecture, and when a user triggers a data request instruction, the client will respond and generate a data request. The data request instruction may be a button click trigger instruction, or may be a voice recognition trigger instruction or other different types of data request instructions. For example, when a user says to the client "retrieve picture A taken last Friday afternoon and compress it", the client will respond to the voice recognition trigger instruction and generate a data request, wherein the data request includes a data processing requirement for compressing the picture and a key that matches picture A taken last Friday afternoon.
[0119] Furthermore, the client sends the data request to the cache node, so that the cache node determines a plug-in that matches the data processing requirement based on the data processing requirement, and determines the corresponding value as the data to be processed based on the key that matches the data requested by the user, and uses the plug-in to process the data to be processed, thereby obtaining the final target data, which is then returned to the client by the cache node.
[0120] Furthermore, when the client receives the target data returned by the cache node, it can provide the target data to the user, which can be done by sounding a reminder to remind the user to check the target data, by sending a text reminder message, or by vibration of the client.
[0121] Specifically, in a possible implementation, please refer to Figure 6 , Figure 6 : is a schematic diagram of the structure of a distributed cache key-value pair architecture registration plug-in provided in an embodiment of the present application. Before generating a data request in response to a data request instruction triggered by a user, the method further includes:
[0122] In response to a plug-in registration instruction triggered by a user, register the plug-in in the registration interface;
[0123] Based on all plug-ins that have been registered with the registration interface, registration plug-in information is generated and sent to the cache node.
[0124] In an embodiment of the present application, before responding to a data request instruction triggered by a user and generating a data request, a plug-in may be registered for the distributed cache key-value pair architecture to complete initialization.
[0125] Specifically, in response to the plug-in registration instruction triggered by the user, the plug-in is registered in the registration interface. It is understandable that when the user triggers the plug-in registration instruction, the user may actively select different types of plug-ins or customize the plug-in by the user, and register the plug-in in the registration interface on the client, so as to meet the user's usage requirements for different scenarios and help improve the user's usage experience.
[0126] Furthermore, when the client's registration interface determines that all plug-ins are registered, it can generate registration plug-in information based on all plug-ins that have been registered in the registration interface and send it to the cache node, so that the cache node can register and preload all plug-ins that have been registered in the registration interface on the client according to the registration plug-in information, thereby completing the initialization of the cache node, so that the subsequent cache node can call the plug-in for data processing.
[0127] As an optional embodiment, the plug-in includes a cropping sub-plug-in, a splicing sub-plug-in, a compression sub-plug-in, and a custom sub-plug-in.
[0128] It is understandable that the plug-in can be divided into different plug-ins such as a cropping sub-plug-in, a splicing sub-plug-in, a compression sub-plug-in, and a custom sub-plug-in according to different data processing functions.
[0129] Among them, the cropping sub-plug-in can crop the acquired data to be processed according to the specified range, so as to return the target data actually needed by the user; the splicing sub-plug-in can splice multiple segments of acquired data to be processed, or it can first crop them through the cropping sub-plug-in and then splice the obtained cropped segments to obtain the target data actually needed by the user; the compression sub-plug-in can compress the acquired data to be processed according to the specified compression algorithm, so as to obtain the compressed target data; the custom sub-plug-in can be customized by the user according to his own usage needs or actual scenario usage needs in advance. The data processing logic is customized into a custom sub-plug-in, so as to perform custom data processing on the acquired data to be processed, and finally obtain the target data that meets the actual needs of the user.
[0130] Specifically, in a possible implementation manner, the method further includes:
[0131] In response to the triggered plug-in update instruction, the plug-in is updated.
[0132] In an embodiment of the present application, the plug-in update instruction can be actively triggered by the user, or it can be generated by the cache node and sent to the client after a certain time interval. When the client responds to the triggered plug-in update instruction, it can check whether all registered plug-ins need to be updated based on the current registered plug-in version, and after all plug-ins are updated in the registration interface, the registered plug-in information can be regenerated and sent to the cache node, thereby updating the plug-in.
[0133] Below, combined with the specific application implementation process, the distributed cache key-value pair architecture data reading method provided in this application is described in detail and explained:
[0134] In an embodiment of the present application, a distributed cache key-value pair architecture data reading method is provided, which can be applied in the field of information processing technology, by receiving a data request sent by a terminal; wherein the data request includes a data processing requirement and a key that matches the data requested by the user; according to the key, a value corresponding to the key is determined as the data to be processed; according to the data processing requirement, a plug-in that matches the data processing requirement is determined; through the plug-in, the target data is read from the data to be processed; and the target data is sent to the terminal. The present application utilizes a plug-in loaded at a cache node to read the data obtained by key-value pair search according to a target range, so that the actual data content required by the user can be directly returned, which can reduce the data processing burden of the terminal, reduce unnecessary data bandwidth occupancy, and improve the performance of the distributed cache architecture, thereby improving the user experience.
[0135] Specifically, please refer to Figure 7 , Figure 7It is a structural diagram of another distributed cache key-value pair architecture provided in an embodiment of the present application. When a user triggers a data request to a client, the client will receive and forward the data request to a cache node.
[0136] Furthermore, after receiving the data request sent by the client, the cache node can determine the matching plug-in according to the data processing requirements included in the data request, and determine the corresponding value as the data to be processed according to the key included in the data request that matches the data requested by the user. The plug-in can be the plug-in information sent by the registration interface on the client received by the cache node during the initialization phase of the distributed cache key-value pair architecture, and register and preload all plug-ins registered by the registration interface on the client according to the registration plug-in information.
[0137] Furthermore, the plug-in is called to perform data processing on the data to be processed. Exemplarily, the data to be processed may be customized by calling a custom sub-plug-in, or the data to be processed may be compressed by calling a compression sub-plug-in.
[0138] Finally, the final target data is obtained and returned to the client so that the client can provide the target data to the user.
[0139] Please refer to Figure 8 , Figure 8 8 is a schematic diagram of a distributed cache key-value pair architecture data reading device provided in an embodiment of the present application. The distributed cache key-value pair architecture data reading device 800 includes:
[0140] The request receiving unit 810 is used to receive a data request sent by a terminal; wherein the data request includes a data processing requirement and a key matching the data requested by the user;
[0141] A data determination unit 820, configured to determine, according to the key, a value corresponding to the key as data to be processed;
[0142] A plug-in determining unit 830 is used to determine a plug-in matching the data processing requirement according to the data processing requirement;
[0143] The data reading unit 840 is used to read the target data from the data to be processed through the plug-in; wherein the target data is data within a target range in the data to be processed;
[0144] The data sending unit 850 is configured to send the target data to the terminal.
[0145] It can be understood that the contents of the above-mentioned distributed cache key-value pair architecture data reading method embodiment are all applicable to the present device embodiment, the functions specifically implemented by the present device embodiment are the same as those of the above-mentioned method embodiment, and the beneficial effects achieved are also the same as those achieved by the above-mentioned method embodiment.
[0146] Please refer to Fig. 9 , Fig. 9 1 is a schematic diagram of another distributed cache key-value pair architecture data reading device provided in an embodiment of the present application. The distributed cache key-value pair architecture data reading device 900 includes:
[0147] A request generating unit 910, configured to generate a data request in response to a triggered data request instruction;
[0148] The request sending unit 920 is used to send the data request to the cache node; wherein the data request includes a data processing requirement and a key matching the data requested by the user;
[0149] The data receiving unit 930 is used to receive the target data returned by the cache node; wherein the target data is read by the cache node from the data to be processed through a plug-in; the plug-in is determined by the cache node according to the data processing requirements; the data to be processed is determined by the cache node according to the key.
[0150] It can be understood that the contents of the above-mentioned other distributed cache key-value pair architecture data reading method embodiment are all applicable to the present device embodiment, the functions specifically implemented by the present device embodiment are the same as those of the above-mentioned method embodiment, and the beneficial effects achieved are also the same as those achieved by the above-mentioned method embodiment.
[0151] Please refer to Fig.10 , Fig.10 The hardware structure of the electronic device provided by the present application is illustrated, and the electronic device includes:
[0152] The processor 1001 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;
[0153] The memory 1002 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1002 can store an operating system and other applications. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1002, and the processor 1001 calls and executes the distributed cache key-value pair architecture data reading method of the embodiment of this application;
[0154] Input / output interface 1003, used to implement information input and output;
[0155] The communication interface 1004 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);
[0156] A bus 1005 , which transmits information between various components of the device (e.g., the processor 1001 , the memory 1002 , the input / output interface 1003 , and the communication interface 1004 );
[0157] The processor 1001 , the memory 1002 , the input / output interface 1003 and the communication interface 1004 are connected to each other in communication within the device via the bus 1005 .
[0158] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.
[0159] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the above-mentioned distributed cache key-value pair architecture data reading method.
[0160] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiments, the functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.
[0161] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0162] The distributed cache key-value pair architecture data reading method, device, equipment and medium provided in the embodiment of the present application receive a data request sent by a terminal; wherein the data request includes a data processing requirement and a key that matches the data requested by the user; according to the key, determine the value corresponding to the key as the data to be processed; according to the data processing requirement, determine a plug-in that matches the data processing requirement; through the plug-in, read the target data from the data to be processed; send the target data to the terminal. The present application uses a plug-in loaded at a cache node to read the data obtained by the key-value pair search according to the target range, so that the actual data content required by the user can be directly returned, which can reduce the data processing burden of the terminal, reduce unnecessary data bandwidth occupation, and improve the performance of the distributed cache architecture, thereby improving the user experience.
[0163] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.
[0164] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.
[0165] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0166] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.
[0167] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0168] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0169] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0170] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0171] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.
[0172] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.
Claims
1. A distributed cache key-value pair architecture data reading method, characterized in that: The method comprises the following steps: Receiving a data request sent by a terminal; wherein the data request includes a data processing requirement and a key matching the data requested by the user; According to the key, determining a value corresponding to the key as data to be processed; According to the data processing requirement, determining a plug-in matching the data processing requirement; By means of the plug-in, target data is read from the data to be processed; wherein the target data is data within a target range in the data to be processed; The target data is sent to the terminal.
2. The distributed cache key-value pair architecture data reading method according to claim 1, characterized in that: Before receiving the data request sent by the terminal, the method further includes: Receiving registration plug-in information sent by a registration interface on the terminal; wherein the registration plug-in information includes all plug-ins that have been registered by the registration interface on the terminal; According to the registered plug-in information, all the plug-ins registered through the registration interface on the terminal are registered and preloaded.
3. The distributed cache key-value pair architecture data reading method according to claim 2, characterized in that: The plug-ins include a cropping sub-plug-in, a splicing sub-plug-in, a compression sub-plug-in and a custom sub-plug-in.
4. The distributed cache key-value pair architecture data reading method according to claim 1, characterized in that: The step of determining, according to the key, a value corresponding to the key as data to be processed includes: Retrieve all key-value pair information cached in the cache node; When key-value pair information related to the key is matched in all the key-value pair information, the value in the key-value pair information is read as the to-be-processed data corresponding to the key.
5. The distributed cache key-value pair architecture data reading method according to claim 4, characterized in that: The method further comprises: When no key-value pair information related to the key is matched in all the key-value pair information, the key is sent to the underlying storage system so that the underlying storage system searches for a corresponding value based on the key; The value sent by the underlying storage system is received as the data to be processed.
6. The distributed cache key-value pair architecture data reading method according to claim 5, characterized in that: After receiving the value sent by the underlying storage system as the data to be processed, the method further includes: Combining the key with the value sent by the underlying storage system into a set of key-value pair information; A set of the key-value pair information is cached in the cache node.
7. The distributed cache key-value pair architecture data reading method according to claim 5, characterized in that: The underlying storage system is a Ceph distributed storage system.
8. A distributed cache key-value pair architecture data reading method, characterized in that: The method comprises: generating a data request in response to a triggered data request instruction; Sending the data request to a cache node; wherein the data request includes a data processing requirement and a key matching the data requested by the user; Receive target data returned by the cache node; wherein the target data is read by the cache node from the data to be processed through a plug-in; the plug-in is determined by the cache node according to the data processing requirements; the data to be processed is determined by the cache node according to the key.
9. The distributed cache key-value pair architecture data reading method according to claim 8, characterized in that: Before generating the data request in response to the data request instruction triggered by the user, the method further includes: In response to a plug-in registration instruction triggered by a user, register the plug-in in the registration interface; Based on all the plug-ins that have been registered with the registration interface, registration plug-in information is generated and sent to the cache node.
10. According to the distributed cache key-value pair architecture data reading method according to claim 9, the plug-in includes a clipping sub-plug-in, a splicing sub-plug-in, a compression sub-plug-in and a custom sub-plug-in.
11. The distributed cache key-value pair architecture data reading method according to claim 8, characterized in that: The method further comprises: In response to the triggered plug-in update instruction, the plug-in is updated.
12. A distributed cache key-value pair architecture data reading device, characterized in that: The device comprises: A request receiving unit, configured to receive a data request sent by a terminal; wherein the data request includes a data processing requirement and a key matching the data requested by the user; a data determination unit, configured to determine, according to the key, a value corresponding to the key as data to be processed; A plug-in determining unit, configured to determine a plug-in matching the data processing requirement according to the data processing requirement; A data reading unit, used to read target data from the data to be processed through the plug-in; wherein the target data is data within a target range in the data to be processed; A data sending unit is used to send the target data to the terminal.
13. A distributed cache key-value pair architecture data reading device, characterized in that: The device comprises: A request generating unit, configured to generate a data request in response to a triggered data request instruction; A request sending unit, used to send the data request to a cache node; wherein the data request includes a data processing requirement and a key matching the data requested by the user; A data receiving unit is used to receive target data returned by the cache node; wherein the target data is read by the cache node from the data to be processed through a plug-in; the plug-in is determined by the cache node according to the data processing requirements; the data to be processed is determined by the cache node according to the key.
14. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the distributed cache key-value pair architecture data reading method described in any one of claims 1 to 11 when executing the computer program.
15. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the distributed cache key-value pair architecture data reading method described in any one of claims 1 to 11 is implemented.
