Data request processing method and device, equipment and storage medium

Abstract

The embodiment of the invention provides a data request processing method and device, equipment and a storage medium. The method comprises the following steps: receiving a deduplicated data request sent by at least one client; according to the data request classification and the request message content, determining a corresponding similar request set from the de-duplicated data requests, the similar request set comprising at least two similar data requests; and combining each similar request set into a single data request, and obtaining a to-be-responded target data request in combination with non-similar requests in the data requests subjected to duplicate removal. According to the embodiment of the invention, the execution redundancy of the duplicated data requests and the similar data requests can be eliminated, and on the basis of ensuring the response accuracy of the data requests, only one response operation needs to be executed on the duplicated data requests and the similar data requests, so that the calculation overhead during data request response is reduced, the response time of the data requests is shortened, and the efficiency is improved. And timely response of the data request is ensured.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of data processing technology, and specifically to a method, device, equipment and storage medium for processing data requests. Background Art

[0002] In various system projects with separated front-end and back-end, the most real and perceptible experience for users is usually the interaction experience between the system and the user. The client will generate corresponding data requests under various user operations and send the data requests to the server, so that the server will respond to the client with the corresponding data, thereby realizing data interaction between the client and the server.

[0003] Currently, the server can receive multiple data requests sent by different clients, and for each data request, the server can use the server data cache method to quickly and efficiently determine the response data corresponding to each data request and return it to the client. However, when there are too many data requests, executing the response processing of each data request in turn will cause a huge computing overhead, increase the response time of each data request, and cannot guarantee a timely response to each data request. Summary of the invention

[0004] The embodiments of the present application provide a method, apparatus, device and storage medium for processing data requests, which eliminate response redundancy of duplicate data requests and similar data requests, reduce the computing overhead when responding to data requests, and ensure timely response to data requests.

[0005] In a first aspect, an embodiment of the present application provides a method for processing a data request, the method comprising:

[0006] receiving a deduplicated data request sent by at least one client;

[0007] Determine a corresponding similar request set from the deduplicated data requests according to the data request classification and the request message content, wherein the similar request set includes at least two similar data requests;

[0008] Each of the similar request sets is merged into a single data request, and combined with the non-similar requests in the deduplicated data requests to obtain a target data request to be responded to.

[0009] In a second aspect, an embodiment of the present application provides a data request processing device, the device comprising:

[0010] A request receiving module, used for receiving a deduplicated data request sent by at least one client;

[0011] A similar request determination module, configured to determine a corresponding similar request set from the deduplicated data requests according to the data request classification and the request message content, wherein the similar request set includes at least two similar data requests;

[0012] The module for determining requests to be responded is used to merge each of the similar request sets into a single data request, and combine the non-similar requests in the deduplicated data requests to obtain a target data request to be responded.

[0013] In a third aspect, an embodiment of the present application provides an electronic device, the electronic device comprising:

[0014] A processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the data request processing method provided in the first aspect of the present application.

[0015] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium for storing a computer program, wherein the computer program enables a computer to execute a method for processing a data request as provided in the first aspect of the present application.

[0016] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the method for processing data requests provided in the first aspect of the present application.

[0017] The embodiment of the present application provides a method, device, equipment and storage medium for processing data requests. Each client will first deduplicate its own data request. Then, when the server receives the deduplicated data requests sent by each client, it will determine the similar data requests in the deduplicated data requests according to the classification and message content of each data request to form a corresponding similar request set. Then, each similar request set is merged into a single data request, and combined with the non-similar requests in the deduplicated data requests, the target data request to be responded is obtained, thereby eliminating the execution redundancy of duplicate data requests and similar data requests. On the basis of ensuring the accuracy of the response to the data request, only one response operation needs to be performed on the duplicate data request and the similar data request to reduce the computational overhead when responding to the data request, reduce the response time of the data request, and ensure the timely response of the data request. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 A flowchart of a method for processing a data request shown in an embodiment of the present application;

[0020] Figure 2 A flowchart of another method for processing a data request provided in an embodiment of the present application;

[0021] Figure 3 A schematic diagram of the principle of the data request processing process provided in the embodiment of the present application;

[0022] Figure 4 A principle block diagram of a data request processing device shown in an embodiment of the present application;

[0023] Figure 5 A schematic block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0026] In order to solve the problem that the server receives too many data requests from the client, and when executing the response processing of each data request in turn, it will cause great computing overhead, the embodiment of the present application designs a new data request processing scheme. When each client generates a corresponding data request, it will first deduplicate its own data request. Then, when the server receives the deduplicated data requests sent by each client, it will determine a similar request set consisting of at least two similar data requests from the deduplicated data requests according to the classification and message content of each data request. Then, each similar request set is merged into a single data request, and combined with the non-similar requests in the deduplicated data request, the target data request to be responded is obtained, thereby eliminating the execution redundancy of duplicate data requests and similar data requests. On the basis of ensuring the accuracy of the response to the data request, only one response operation needs to be performed on the duplicate data request and the similar data request to reduce the computing overhead when responding to the data request, reduce the response time of the data request, and ensure the timely response of the data request.

[0027] Figure 1 This is a flow chart of a method for processing a data request according to an embodiment of the present application. Figure 1 , the method may include the following steps:

[0028] S110, receiving a deduplicated data request sent by at least one client.

[0029] In various system projects with separated front-end and back-end, the most real and perceptible experience for users is usually the interactive experience between the system and the user. Usually, when the client detects various user-triggered operations (such as form filling operations on a certain page of the client, data search operations, etc.), or the client performs a data interaction operation with the server under automatic scheduling (such as the client regularly feeds back the latest background data to the user, etc.), it can be said that the client has corresponding interactive needs for the corresponding background data of the server. Therefore, in response to the above situation, the client usually generates a corresponding data request and sends it to the server, so that the server responds to the client with the corresponding data to realize data interaction between the client and the server.

[0030] Considering that each client may generate multiple data requests per unit time, some of these data requests may be identical, resulting in a large number of identical data requests arriving at the server at the same time per unit time. Then, under high load, the server needs to process a large number of identical data requests from various clients, which wastes the server's computing power and affects the timely response of data requests.

[0031] In order to solve the above problem, for each data request generated by each client within a unit time, the client will first parse the actual message content of each data request within the unit time locally to determine whether there is an identical data request within the unit time, and perform a deduplication process on each data request within the unit time, thereby obtaining the deduplicated data request of each client within the unit time.

[0032] In some implementable methods, for the same data request within a unit time, the present application can make a judgment in the following manner: for each data request within a unit time, the actual message content of the data request can be parsed to obtain the request address and various request parameters of the data request. Then, the request address and various request parameters of each data request are encrypted using a preset information encryption algorithm to obtain the corresponding encrypted information as the identification information of the data request. Then, according to the above method, the identification information of each data request within a unit time can be obtained, so that multiple data requests with the same identification information can be regarded as the same data request within a unit time.

[0033] In this application, after each client deduplicates each data request within a unit time, it can send the deduplicated data request to the server. Then, the server can receive the deduplicated data request generated by each client within a unit time, so as to avoid the situation where the server repeatedly processes too many identical data requests under high load and causes a waste of computing power.

[0034] It is understandable that, considering that the same client may generate the same data request within a unit time, different clients may also generate the same data request within a unit time. Therefore, in order to ensure the uniqueness of the server's processing of data requests within a unit time, after receiving the deduplicated data requests sent by each client, the server can also aggregate the deduplicated data requests sent by each client and perform secondary deduplication on the aggregated data requests.

[0035] That is to say, after receiving the deduplicated data requests sent by each client, the server can aggregate each deduplicated data request sent by each client. Then, by judging whether the identification information of each aggregated data request is the same, it is judged whether the same data request still exists in the aggregated data request, and the aggregated data request is deduplicated twice to obtain the final deduplicated data request, thereby ensuring that each data request processed by the server in a unit time is unique, and avoiding the server from repeatedly processing too many identical data requests under high load, thereby reducing the computational overhead and data request response time when responding to data requests.

[0036] S120, determining a corresponding similar request set from the deduplicated data requests according to the data request classification and the request message content, where the similar request set includes at least two similar data requests.

[0037] Considering that data requests can be divided into multiple types according to different data operations, different specific execution methods can be selected for data requests under different categories. For example, data requests under certain categories can only ensure accurate responses to data requests when they are executed sequentially. Data requests under other categories can support either sequential execution or batch execution, both of which can ensure accurate responses to data requests.

[0038] Therefore, in order to ensure timely response to data requests, after obtaining the deduplicated data requests, the server will first determine the type of each data request. Then, by analyzing whether each data request classification supports batch execution of data requests, some data requests under certain classifications that can support batch execution are screened out from the deduplicated data requests. Furthermore, by parsing the actual message content of each screened data request, it is determined whether there are at least two data requests that operate on the background data at the same address in the database, and at least two data requests that perform the same operation on the background data at the same address in the database are taken as similar data requests to form a corresponding similar request set. Thus, by performing the above steps on the deduplicated data requests, multiple similar request sets can be obtained, and each similar request set can include at least two similar data requests to represent operations on certain background data at the same address in the database.

[0039] It is understandable that data requests can usually be divided into requests for four data operations: adding, deleting, modifying, and querying the background data of the server. Among them, the relevant data requests for the three data operations of adding, deleting, and modifying usually require substantial modifications to the background data stored in the database of the server, so that different data requests need to modify different data in the database, which means that they need to be executed in sequence on the server to achieve accurate response. The data query request only queries certain background data stored in the database of the server, and does not modify the data in the database. Moreover, for multiple data query requests, corresponding batch query interfaces can usually be provided in the service development stage to aggregate and distribute various query logics, so as to support batch query of response data corresponding to multiple data query requests of the same type, and do a good job of identifying the response results of batch query data dispersed into a single data query request.

[0040] From this, it can be seen that for the similar request set in the deduplicated data requests, the present application can be determined in the following manner: according to the data request classification, the corresponding data query request is determined from the deduplicated data requests; the request message content of each data query request is parsed, and the data query requests for the same request address are combined into a corresponding similar request set.

[0041] That is to say, for the deduplicated data requests, the server can first determine the type of each data request to filter out the corresponding data query request from the deduplicated data requests, and confirm that data requests under other categories do not support batch execution. Then, by parsing the actual message content of each data query request, the request address and various request parameters of each data query request are determined. At this point, the data query request under the same request address can represent a query for part of the data corresponding to different request parameters under the same request address in the database.

[0042] Therefore, the server can classify each data query request into the corresponding request address. If there are multiple data query requests under a request address, the multiple data query requests under the request address can be combined into a similar request set. If there is only one data query request under a request address, it means that there is no similar data request for the data query request, and the data query request is regarded as a non-similar request.

[0043] S130, merging each similar request set into a single data request, and combining the non-similar requests in the deduplicated data requests to obtain a target data request to be responded to.

[0044] Since each similar data request in each similar request set can operate on the corresponding data under the same request address, batch operations on the corresponding data are supported. Then, in order to ensure efficient response to data requests, for each similar request set, the server can parse the actual message content of each data request in the similar request set to obtain the request address and various request parameters of each data request in the similar request set, and merge the various request parameters of each data request in the similar request set together, and combine the same request address of each data request in the similar request set to form a new data request message content, which can be a single data request that merges the various data requests in the similar request set.

[0045] It can be understood that the single data request after merging each similar request set will also mark the associated information of the request parameters of each data request in the similar request set, so that after the entire response data is obtained in batches through the merged data requests, it can be accurately dispersed into single response data for each data request in the similar request set, thereby achieving accurate response to the data request.

[0046] Then, similar data requests in each similar request are excluded from the deduplicated data requests, and the remaining data requests in the deduplicated data requests are used as corresponding non-similar requests. Then, the new data requests after merging each similar request set and each non-similar request are aggregated together to obtain the target data request to be responded to, so that each target data request can be executed in sequence later to eliminate the execution redundancy of duplicate data requests and similar data requests, and only one response operation needs to be performed on duplicate data requests and similar data requests to reduce the response time of data requests.

[0047] In the technical solution provided by the embodiment of the present application, each client will first deduplicate its own data request. Then, when the server receives the deduplicated data requests sent by each client, it will determine the similar data requests in the deduplicated data requests according to the classification and message content of each data request to form a corresponding similar request set. Then, each similar request set is merged into a single data request, and combined with the non-similar requests in the deduplicated data requests, the target data request to be responded to is obtained, thereby eliminating the execution redundancy of duplicate data requests and similar data requests. On the basis of ensuring the accuracy of the response to the data request, only one response operation needs to be performed on the duplicate data request and the similar data request to reduce the computational overhead when responding to the data request, reduce the response time of the data request, and ensure the timely response of the data request.

[0048] As an optional implementation method in this application, in order to ensure accurate response to data requests, after receiving the target data request to be responded to, the server needs to perform response operations for each target data request in sequence. Considering the different classifications of target data requests, the actual operations on specific response data are also different.

[0049] That is, the target data request can be divided into data addition request, data deletion request, data modification request and data query request. Among them, when executing the response, the data addition request, data deletion request and data modification request all need to substantially modify the background data stored in the server's database. When executing the response, the data query request only needs to query the corresponding background data stored in the server's database, without substantially modifying the server's database.

[0050] From this, we can see that the responses to data addition requests, data deletion requests, and data modification requests require the direct participation of the database, while the responses to data query requests only focus on whether the response data in the database can be accurately obtained, and do not focus on whether it is obtained directly from the database or from other locations where the corresponding data in the database is cached.

[0051] Then, in order to ensure efficient response to data requests, for target data requests to be responded to, the server can use different response methods related to the database according to the classification of the target data requests to perform the response operation of each target data request. That is, for some target data requests that require direct participation of the database, the server can complete the data response of the target data request by directly performing corresponding data interaction operations on the database. For other target data requests that do not require direct participation of the database, the server can complete the data response of the target data request by performing corresponding data interaction operations through a specific cache service used to cache the corresponding data in the database. Accordingly, for target data requests of different classifications, different response methods can be used to perform specific response operations, thereby ensuring accurate responses to target data requests.

[0052] In some possible implementations, in order to ensure the accuracy of the target data request, the present application may provide a detailed explanation of the specific response process of target data requests of different categories.

[0053] Figure 2 A flowchart of another method for processing a data request provided in an embodiment of the present application. Figure 3 As shown, the method may specifically include the following steps:

[0054] S210, receiving a deduplicated data request sent by at least one client.

[0055] S220, determining a corresponding similar request set from the deduplicated data requests according to the data request classification and the request message content, where the similar request set includes at least two similar data requests.

[0056] S230, merging each similar request set into a single data request, and combining the non-similar requests in the deduplicated data requests to obtain a target data request to be responded to.

[0057] S240, determining whether the target data request is a data query request, if so, executing S250; if not, executing S260.

[0058] In this application, according to the solution provided in the above embodiment, Figure 3As shown in the figure, after multiple deduplication of data requests from various clients and merging similar data requests after deduplication into a single data request, the target data request to be responded to can be obtained. The target data request can be divided into data addition request, data deletion request, data modification request and data query request. Among them, the response to data addition request, data deletion request and data modification request requires the direct participation of the database, while the response to the data query request only focuses on whether the response data in the database can be accurately obtained, and does not focus on whether it is obtained directly from the database or from other locations where the corresponding data in the database is cached.

[0059] From this, it can be seen that by determining whether the target data request is a data query request, it is possible to determine whether the target data request requires direct participation of the database when responding, and choose to use an appropriate response method to accurately respond to the target data request.

[0060] S250, determining the response data of the target data request through the target distributed node in the distributed cache service, and each distributed node in the distributed cache service updates the corresponding cache data from a preset pre-fetch support library.

[0061] Considering that if all target data requests are executed through direct interaction with the database to perform corresponding response operations, it will cause great data processing pressure on the database, thus affecting the response efficiency of data requests. Therefore, the server can set up a distributed cache service to cache the corresponding data in the database according to a certain cache strategy to support the corresponding response processing of data query requests.

[0062] Among them, Figure 3 As shown, the distributed cache service can be composed of multiple distributed nodes, and each distributed node can obtain different data from the database for caching according to a certain cache strategy, so that different distributed nodes can execute data responses for different data query requests.

[0063] Specifically, it is considered that the database not only needs to support the data reading function of the distributed cache service, but also needs to support the function of users writing data. In this case, when the database supports a large business system, it may need to process a large number of data reading operations and data writing operations at the same time, which is very likely to cause data errors or crashes in the database. Then, in order to ensure the successful processing of data requests, this application can use the database that originally stores background data on the server side as the corresponding main database, and on the basis of the main database, additionally configure a pre-fetch support library to separate and copy all the data in the main database in a real-time synchronization manner, and store it in the pre-fetch support library, so as to share the pressure of the main database. Among them, real-time data synchronization can be maintained between the main database and the pre-fetch support library, that is, the main database will synchronize the data changes within it to the pre-fetch support library in real time to ensure that the pre-fetch support library can be consistent with the main database in real time.

[0064] Furthermore, in order to ensure accurate response to data requests, each distributed node in the distributed cache service can update the corresponding cache data from the preset pre-fetch support library. That is, each distributed node in the distributed cache service can obtain the corresponding background data from the pre-fetch support library for caching according to a certain cache strategy, so that different distributed nodes can execute data responses for different data query requests.

[0065] Therefore, for the target data request to be responded to, if the target data request is a data query request, the server will first analyze which part of the data the target data request needs to query, and which distributed node this part of the data is cached on, to determine the target distributed node corresponding to the target data request from the various distributed nodes of the distributed cache service. Then, the target data request is distributed to the target distributed node, so that the target distributed node executes the target data request to determine the response data of the target data request from the latest data cached by the target distributed node.

[0066] In some possible implementations, in order to ensure the cache accuracy of the distributed cache service, each distributed node can be divided into multiple distributed node pairs. The two distributed nodes in each distributed node pair can serve as primary and backup nodes to cache the same data in the database to prevent problems with a distributed node from causing data requests to fail to respond, thereby ensuring successful responses to data requests as much as possible.

[0067] Moreover, in order to ensure the accurate response of the target data request, it is first necessary to ensure the accuracy of the target distributed node used to execute the target data request. In this regard, the server can first determine the allocation of the target data request to each distributed node in the distributed cache service before executing a specific response operation for each target data request according to the specific classification and quantity of the target data request in a unit time, so as to determine the corresponding relationship between each target data request and each distributed node in the distributed cache service.

[0068] Specifically, at the beginning of the backend service for data request processing, such as Figure 3 As shown, first, the backend service corresponding to each target data request will be registered in the corresponding registration center, so that the registration center can apply for all target data requests in the backend service. It can be seen that in the real-time processing of target data requests, the registration center can apply for and obtain all target data requests. Then, the server can read all target data requests from the registration center, and classify the target data requests to determine the data query requests therein, and determine the specific distribution of each data query request to each distributed node according to the specific category of the data queried by each data query request and the specific number of data query requests under each category, so that the amount of cache tasks allocated to each distributed node is fixed, avoiding the situation where the cache pressure of a distributed node is too large.

[0069] It should be noted that with the continuous successful response to the target data request and the continuous generation of new target data requests, the target data request of the registration center will be continuously updated, so that the server can continuously read the new target data request to continuously update the request allocation information of each distributed node.

[0070] Then, every time the server updates the request allocation information of each distributed node, Figure 3 As shown, the latest request allocation information will be sent to the distributed cache service to analyze the latest cache data of each distributed node. Moreover, considering that the call interval of a data query request is long, it means that the response data of the last call of the data query request may still be cached on the corresponding distributed node, and during the period when the data query request is not called, the data related to the data query request in the database may change, so that the cache data of the distributed node corresponding to the data query request is not updated. It can be seen that the call interval of the data query request and the request allocation information of each distributed node will affect the update of the cache data of each distributed node in the distributed cache service.

[0071] Therefore, in order to ensure accurate caching of the distributed cache service, the server can update the cache data of each distributed node in the distributed cache service from the preset pre-fetch support library according to the call interval of the target data request and the request allocation information of each distributed node in the distributed cache service.

[0072] Specifically, each time the server updates the request allocation information of each distributed node, it will analyze the latest data information for the latest data query request assigned to each distributed node, read the corresponding latest cache data from the preset pre-fetch support library, and update the cache data of each distributed node in the distributed cache service. Moreover, when the target data request is a data query request, the server can also analyze whether the call interval of the data query request is long, and combine the latest request allocation information to determine whether the cache data of the distributed node assigned to the data query request needs to be updated, so as to timely update the cache data of each distributed node in the distributed cache service from the preset pre-fetch support library.

[0073] In some implementable methods, the server can plan two types of queues for data query requests according to the call volume of each data query request per unit time, namely: a high-concurrency request queue of the same type and a high-concurrency request queue of different types. Among them, the high-concurrency request queue of the same type can be used to store high-concurrency data query requests of the same type per unit time, and the high-concurrency request queue of different types can be used to store high-concurrency data query requests of different types per unit time.

[0074] Then, for high-concurrency data query requests of different types, in the process of continuous calls of data query requests, the minimum interval of data query request calls can be recorded. When the call interval exceeds a certain threshold, it means that the data query request has not been called for a long time, causing the cache data of the corresponding distributed node to time out and not be updated. Therefore, the data query request can be marked as a data query request with no demand, and the cache data of the distributed node corresponding to the data query request can be cleared to release excess space, waiting for the next call to update the cache data of the distributed node from the preset pre-fetch support library again. For high-concurrency data query requests of the same type, a sliding time window can be used to determine whether to execute the data response through the target distributed node of the data query request, or to update the cache data of the target distributed node of the data query request from the preset pre-fetch support library, for example: use cached data within 1s, and refresh the cache if it exceeds 1s.

[0075] S260, executing a response operation of the target data request in the primary database.

[0076] If the target data request is not a data query request, it means that the target data request is one of a data addition request, a data deletion request, and a data modification request, and the corresponding data in the master database needs to be adjusted accordingly. Therefore, the server can directly perform the corresponding data interaction operation in the master database by parsing the actual data interaction operation (such as data addition, data deletion, and data modification, etc.) represented by the target data request, and execute the specific response operation of the target data request to ensure the accurate response of the data request.

[0077] In some implementable methods, it is considered that the response data for non-data query requests may be background record data or long-term unused data that some users do not pay attention to, or may be real-time interactive data that some users pay attention to. Then, in the case where the target data request is a non-data query request, by analyzing the type of response data for the target data request, the target data request can be divided into real-time requests and non-real-time requests. Among them, the target data request being a real-time request can indicate that the response data for the target data request is real-time interactive data that some users pay attention to, and the target data request being a non-real-time request can indicate that the response data for the target data request is background record data or long-term unused data that some users do not pay attention to.

[0078] Therefore, in order to ensure efficient response to the target data request, when the target data request is not a data query request, the response operation of the target data request executed in the main database can be specifically: if the target data request is a real-time request, the immediate response operation of the target data request is executed in the main database; if the target data request is a non-real-time request, the target data request is written to the preset idle request queue, so as to execute the response operation of the target data request in the main database through the idle request queue.

[0079] That is, when the target data request is a data query request, firstly, by analyzing the type of response data to which the target data request is directed, it is determined whether the target data request is a real-time request or a non-real-time request, and then different response methods are used to execute the response operation of the target data request.

[0080] Specifically, in the case where the target data request is a data query request, if the target data request is a real-time request, it means that the target data request needs to be responded to immediately to avoid the problem that the response data is not adjusted in time due to the delay in the response to the target data request, which causes the subsequent interaction of the response data to be wrong. Figure 3As shown, the server will parse the actual data interaction operation (such as data addition, data deletion and data modification) represented by the target data request to instantly perform corresponding data interaction operations in the main database, and execute the instant response operation of the target data request to ensure timely response to the data request.

[0081] If the target data request is a non-real-time request, it indicates whether the response data to the target data request is adjusted in time, and will not affect the subsequent data interaction. In order to avoid the problem of delayed response to data requests when responding to a large number of data requests in batches, the server can pre-set an idle request queue to store non-real-time requests in the target data request, waiting for subsequent responses, reducing the data request processing pressure on the server. Therefore, when the target data request is a non-real-time request, such as Figure 3 As shown, the target data request can be written into the idle request queue to wait for subsequent responses. At this time, the response operations of each target data request in the idle request queue can be executed in the main database in sequence, and the delayed response of the target data request in the idle request queue will not affect the accuracy of subsequent data interaction, and can also improve the efficient response of data requests.

[0082] The technical solution provided in the embodiment of the present application can select different response methods to execute the response operation of the target data request for the data query request and the non-data query request in the target data request to be responded to, thereby improving the flexibility of the data request response.

[0083] Figure 4 The following is a block diagram of a data request processing device according to an embodiment of the present application. Figure 4 As shown, the apparatus 400 may include:

[0084] The request receiving module 410 is used to receive a deduplicated data request sent by at least one client;

[0085] A similar request determination module 420, configured to determine a corresponding similar request set from the deduplicated data requests according to the data request classification and the request message content, wherein the similar request set includes at least two similar data requests;

[0086] The request-to-be-responded determining module 430 is configured to merge each of the similar request sets into a single data request, and combine the non-similar requests in the deduplicated data requests to obtain a target data request to be responded to.

[0087] In some implementations, the similar request determination module 420 may be specifically configured to:

[0088] According to the data request classification, the corresponding data query request is determined from the deduplicated data requests;

[0089] The request message content of each of the data query requests is parsed, and the data query requests with the same request address are combined into a corresponding similar request set.

[0090] In some implementations, the data request processing device 400 may further include:

[0091] The data request deduplication module is used to aggregate the deduplicated data requests sent by each of the clients and perform secondary deduplication on the aggregated data requests.

[0092] In some implementations, the data request processing device 400 may further include:

[0093] The data request response module is used to execute a response operation of the target data request according to the classification of the target data request.

[0094] In some implementations, the data request response module may include:

[0095] The first type of request response unit is used to determine the response data of the target data request through the target distributed node in the distributed cache service if the target data request is a data query request, and each distributed node in the distributed cache service updates the corresponding cache data from a preset pre-fetch support library;

[0096] The second type of request response unit is used to execute a response operation of the target data request in the main database if the target data request is a non-data query request.

[0097] In some implementations, the data request processing device 400 may further include:

[0098] The data cache module is used to update the cache data of each distributed node in the distributed cache service from a preset pre-fetch support library according to the call interval of the target data request and the request allocation information of each distributed node in the distributed cache service.

[0099] In some implementations, the second type of request response unit may be specifically used for:

[0100] If the target data request is a real-time request, executing an immediate response operation of the target data request in the primary database;

[0101] If the target data request is a non-real-time request, the target data request is written into a preset idle time request queue, so that a response operation of the target data request is executed in the primary database through the idle time request queue.

[0102] In the embodiment of the present application, each client will first deduplicate its own data request. Then, when the server receives the deduplicated data requests sent by each client, it will determine the similar data requests in the deduplicated data requests according to the classification and message content of each data request to form a corresponding similar request set. Then, each similar request set is merged into a single data request, and combined with the non-similar requests in the deduplicated data requests, the target data request to be responded is obtained, thereby eliminating the execution redundancy of duplicate data requests and similar data requests. On the basis of ensuring the accuracy of the response to the data request, only one response operation needs to be performed on the duplicate data request and the similar data request to reduce the computational overhead when responding to the data request, reduce the response time of the data request, and ensure the timely response of the data request.

[0103] It should be understood that the device embodiment and the method embodiment may correspond to each other, and similar descriptions may refer to the method embodiment. To avoid repetition, they will not be described here. Specifically, Figure 4 The device 400 shown can execute any method embodiment in the present application, and the aforementioned and other operations and / or functions of each module in the device 400 are respectively for implementing the corresponding processes in each method in the embodiment of the present application. For the sake of brevity, they will not be repeated here.

[0104] The above describes the device 400 of the embodiment of the present application from the perspective of the functional module in conjunction with the accompanying drawings. It should be understood that the functional module can be implemented in hardware form, can be implemented by instructions in software form, and can also be implemented by a combination of hardware and software modules. Specifically, the steps of the method embodiment in the embodiment of the present application can be completed by the hardware integrated logic circuit and / or software form instructions in the processor, and the steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or a combination of hardware and software modules in the decoding processor to perform. Optionally, the software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory, and completes the steps in the above method embodiment in conjunction with its hardware.

[0105] Figure 5 A schematic block diagram of an electronic device according to an embodiment of the present application.

[0106] like Figure 5 As shown, the electronic device 500 may include:

[0107] The memory 510 and the processor 520, the memory 510 is used to store the computer program and transmit the program code to the processor 520. In other words, the processor 520 can call and run the computer program from the memory 510 to implement the method in the embodiment of the present application.

[0108] For example, the processor 520 may be configured to execute the above method embodiments according to instructions in the computer program.

[0109] In some embodiments of the present application, the processor 520 may include but is not limited to:

[0110] General-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc.

[0111] In some embodiments of the present application, the memory 510 includes but is not limited to:

[0112] Volatile memory and / or non-volatile memory. Among them, the non-volatile memory can be read-only memory (ROM), programmable ROM (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM) or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus random access memory (DR RAM).

[0113] In some embodiments of the present application, the computer program may be divided into one or more modules, which are stored in the memory 510 and executed by the processor 520 to complete the method provided by the present application. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program in the electronic device.

[0114] like Figure 5 As shown, the electronic device may also include:

[0115] The transceiver 530 may be connected to the processor 520 or the memory 510 .

[0116] The processor 520 may control the transceiver 530 to communicate with other devices, specifically, to send information or data to other devices, or to receive information or data sent by other devices. The transceiver 530 may include a transmitter and a receiver. The transceiver 530 may further include an antenna, and the number of antennas may be one or more.

[0117] It should be understood that the various components in the electronic device are connected via a bus system, wherein the bus system includes not only a data bus but also a power bus, a control bus and a status signal bus.

[0118] The present application also provides a computer storage medium on which a computer program is stored, and when the computer program is executed by a computer, the computer can perform the method of the above method embodiment. In other words, the present application embodiment also provides a computer program product containing instructions, and when the instructions are executed by a computer, the computer can perform the method of the above method embodiment.

[0119] When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integration. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a digital video disc (digital video disc, DVD)), or a semiconductor medium (e.g., a solid state drive (solid state disk, SSD)), etc.

[0120] Those of ordinary skill in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0121] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the module is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0122] The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. For example, each functional module in each embodiment of the present application may be integrated into a processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0123] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for processing a data request, characterized in that: include: receiving a deduplicated data request sent by at least one client; Determine a corresponding similar request set from the deduplicated data requests according to the data request classification and the request message content, wherein the similar request set includes at least two similar data requests; Each of the similar request sets is merged into a single data request, and combined with the non-similar requests in the deduplicated data requests to obtain a target data request to be responded to.

2. The method according to claim 1, characterized in that The step of determining a corresponding similar request set from the deduplicated data requests according to the data request classification and the request message content includes: According to the data request classification, the corresponding data query request is determined from the deduplicated data requests; The request message content of each of the data query requests is parsed, and the data query requests with the same request address are combined into a corresponding similar request set.

3. The method according to claim 1, characterized in that The method further comprises: The deduplicated data requests sent by the clients are aggregated, and the aggregated data requests are deduplicated a second time.

4. The method according to claim 1, characterized in that The method further comprises: According to the classification of the target data request, a response operation of the target data request is performed.

5. The method according to claim 4, characterized in that The step of performing a response operation of the target data request according to the classification of the target data request includes: If the target data request is a data query request, the response data of the target data request is determined through the target distributed node in the distributed cache service, and each distributed node in the distributed cache service updates the corresponding cache data from a preset pre-fetch support library; If the target data request is a non-data query request, a response operation of the target data request is executed in the primary database.

6. The method according to claim 5, characterized in that When the target data request is a data query request, the method further includes: According to the calling interval of the target data request and the request allocation information of each distributed node in the distributed cache service, the cache data of each distributed node in the distributed cache service is updated from a preset pre-fetch support library.

7. The method according to claim 5, characterized in that The executing the response operation of the target data request in the primary database includes: If the target data request is a real-time request, executing an immediate response operation of the target data request in the primary database; If the target data request is a non-real-time request, the target data request is written into a preset idle time request queue, so that a response operation of the target data request is executed in the primary database through the idle time request queue.

8. A data request processing device, characterized in that: include: A request receiving module, used for receiving a deduplicated data request sent by at least one client; A similar request determination module, configured to determine a corresponding similar request set from the deduplicated data requests according to the data request classification and the request message content, wherein the similar request set includes at least two similar data requests; The module for determining requests to be responded is used to merge each of the similar request sets into a single data request, and combine the non-similar requests in the deduplicated data requests to obtain a target data request to be responded.

9. An electronic device, characterized in that: include: A processor and a memory, the memory being used to store a computer program, and the processor being used to call and run the computer program stored in the memory to execute the method for processing a data request according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein the computer program enables a computer to execute the method for processing a data request according to any one of claims 1 to 7.

Images