Data batch request method and apparatus, computer device, and storage medium
By retrieving and updating cached data, the inefficiency caused by cached data invalidation in batch caching is solved, and synchronous refresh of the cache and database is achieved, thereby improving data reading efficiency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA PING AN PROPERTY INSURANCE CO LTD
- Filing Date
- 2023-09-15
- Publication Date
- 2026-06-26
AI Technical Summary
In existing batch caching methods, the hit rate of cached data is not high, resulting in low data reading efficiency. In particular, when cached data expires, frequent database queries are required, which affects system performance.
By obtaining batch request instructions, the corresponding batch request parameters are obtained, and valid request parameters are saved according to preset parameter saving information; the return result of cached data is checked. If it is not empty, the cached data is updated. If it is empty, the parameters to be requested are filtered and sent to the database to reduce the number of database accesses.
It improves the efficiency of batch data requests, ensures that cached data is synchronized with the database, reduces the number of database queries, and improves data reading speed.
Smart Images
Figure CN117194496B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing technology, specifically to the field of financial technology, and particularly to a method, apparatus, computer device, and storage medium for batch data requests. Background Technology
[0002] Caching is a common solution for handling high-concurrency scenarios and alleviating database read / write pressure. It involves storing frequently used data in a faster data cache (such as memory), and then retrieving the data from the cache using request parameters. Caching effectively speeds up data retrieval during requests, thus improving overall system performance.
[0003] Batch caching is a caching method that caches data in batches. It allows for the retrieval of multiple data sets of the same type within a short period, improving data processing efficiency. For example, in a product sales platform dealing with large amounts of sales data, batch caching allows for the simultaneous retrieval of large volumes of data, effectively enhancing the platform's query and processing efficiency. Batch caching works by querying the cached data corresponding to each request parameter in the cache and reading the retrieved data all at once to obtain a batch of cached data.
[0004] Currently, batch caching involves assigning a cached data set to each parameter in a batch. After sending the batch parameters to the data cache, the corresponding cached data is queried individually for each parameter, and then all cached data is returned together. However, this approach has several drawbacks. First, when querying the data cache, some parameters may have corresponding cached data, while others may not. In this case, the system will send the parameters without corresponding cached data to the database for querying and directly read the retrieved data, thus slowing down data retrieval. Second, because the validity period of the cached data for each parameter is different, many requests may be sent after the cached data has expired. This forces the system to still query the database for expired data, resulting in a low cache hit rate and reduced data retrieval efficiency. Summary of the Invention
[0005] The purpose of this application is to provide a method, apparatus, computer device, and storage medium for batch data requests, so as to solve the problem of low efficiency when making batch data requests.
[0006] To address the aforementioned technical problems, this application provides a method for batch data requests, employing the following technical solution:
[0007] Obtain a batch request instruction, and obtain the corresponding batch request parameters based on the batch request instruction;
[0008] Obtain preset parameter saving information, and save the batch request parameters within a predetermined time period according to the parameter saving information to obtain valid request parameters;
[0009] Request cached data according to the batch request parameters, and check whether the returned result of the cached data is empty;
[0010] If the returned result of the cached data is not empty, then obtain the update information of the cached data, and update the cached data according to the update information to obtain the batch request data; and
[0011] If the cached data returns an empty result, the valid request parameters are filtered according to the update information to obtain the parameters to be requested, and the parameters to be requested are sent to the database to request data, thereby obtaining the batch request data.
[0012] Furthermore, the step of obtaining a batch request instruction and obtaining the corresponding batch request parameters based on the batch request instruction specifically includes:
[0013] Monitor the requesting end, obtain the batch request operation of the requesting end, and generate the batch request instruction based on the batch request operation;
[0014] The batch request instruction is split into separate instruction segments to obtain the request instruction; and
[0015] The corresponding request parameters are obtained according to the request instruction, and the request parameters are merged and organized to obtain the batch request parameters.
[0016] Furthermore, the parameter storage information includes storage time information, storage method information, and storage check information. The step of obtaining a preset parameter storage rule and storing the batch request parameters within a predetermined time period according to the parameter storage information to obtain valid request parameters specifically includes:
[0017] Obtain the storage time information and parse the storage time information to obtain the storage duration;
[0018] Obtain the request time of the batch request parameters, and sort the request times to obtain a request time sorting table;
[0019] The request time closest to the current time in the request time sorting table is used as the starting time for saving;
[0020] The predetermined time period is calculated based on the storage start time and storage duration;
[0021] Obtain the request parameters to be saved corresponding to the predetermined time period from the request time sorting table, and asynchronously save the request parameters to be saved according to the saving method information to obtain the saved request parameters;
[0022] The save check information is used to determine whether the request parameters to be saved were saved successfully.
[0023] If the request parameters to be saved are saved successfully, then the saved request parameters will be used as the valid request parameters; and
[0024] If the request parameters to be saved fail to be saved, the request parameters to be saved are retrieved again and saved again until the request parameters to be saved are saved successfully.
[0025] Furthermore, the step of requesting cached data according to the batch request parameters and detecting whether the returned result of the cached data is empty specifically includes:
[0026] Obtain the batch request parameters and send the batch request parameters to the data cache to query the cached data;
[0027] The cached data is extracted and returned to the requesting end to obtain the return result; and
[0028] Obtain preset null value detection information, and detect whether the returned result is null based on the control detection information.
[0029] Furthermore, the step of obtaining update information for the cached data and updating the cached data according to the update information to obtain batch request data specifically includes:
[0030] Obtain the preset update duration and the request time of the batch request data, and calculate the update period corresponding to the cached data based on the request time and the update duration;
[0031] Monitor the data cache area corresponding to the cached data, and record the operation behavior of the data cache area during the update period, wherein the operation behavior includes at least update behavior;
[0032] The operation behavior is associated with the cached data, and the update information is generated based on the operation behavior;
[0033] Detect whether the update information includes the update behavior;
[0034] If the update information includes the update behavior, then the cached data will be used as the batch request data; and
[0035] If the update information does not include the update behavior, then obtain the data identifier of the cached data, search for the corresponding update cached data in the database according to the data identifier, and update the cached data according to the update cached data.
[0036] Furthermore, the step of filtering the valid request parameters based on the updated information to obtain the parameters to be requested, and sending the parameters to be requested to the database to request data, thereby obtaining the batch request data, specifically includes:
[0037] Obtain the update information and determine the corresponding update request parameters based on the update information;
[0038] Obtain the valid request parameters, filter the valid request parameters according to the update request parameters, and obtain the parameters to be requested; and
[0039] The requested parameters are sent to the database to query the batch request data, and the batch request data is extracted.
[0040] Furthermore, before the step of sending the requested parameters to the database to query the batch request data and extracting the batch request data, the following steps are also included:
[0041] Obtain error information from the database and detect whether the error information reaches a preset threshold;
[0042] If the error message reaches the preset threshold, the database data interface will be changed from an open state to a closed state, and empty data will be returned when the requested parameters are used for data querying; and
[0043] If the error message does not reach the preset threshold, the data interface remains open until the database error message reaches the preset threshold or the database closes the data interface.
[0044] To address the aforementioned technical problems, this application also provides a data batch request device, which employs the following technical solution:
[0045] The parameter acquisition module is used to acquire batch request instructions and obtain corresponding batch request parameters based on the batch request instructions.
[0046] The parameter saving module is used to obtain preset parameter saving information, and save the batch request parameters within a predetermined time period according to the parameter saving information to obtain valid request parameters;
[0047] The data detection module is used to request cached data according to the batch request parameters and to detect whether the returned result of the cached data is empty.
[0048] The first data request module is configured to, if the return result of the cached data is not empty, obtain the update information of the cached data, and update the cached data according to the update information to obtain batch request data; and
[0049] The second data request module is used to filter the valid request parameters according to the update information if the return result of the cached data is empty, obtain the parameters to be requested, and send the parameters to be requested to the database to make a data request, thereby obtaining the batch request data.
[0050] To address the aforementioned technical problems, this application also provides a computer device that employs the following technical solution:
[0051] A computer device includes a memory and a processor, the memory storing computer-readable instructions, the processor executing the computer-readable instructions to implement the steps of the data bulk request method as described in any of the preceding claims.
[0052] To address the aforementioned technical problems, this application also provides a computer-readable storage medium, employing the technical solution described below:
[0053] A computer-readable storage medium storing computer-readable instructions that, when executed by a processor, implement the steps of the data bulk request method as described in any of the preceding claims.
[0054] Compared with the prior art, the embodiments of this application have the following advantages: This embodiment obtains batch request instructions and corresponding batch request parameters according to the batch request instructions, thereby facilitating subsequent querying of cached data based on the batch request parameters; by obtaining preset parameter storage information, the batch request parameters within a predetermined time period are stored according to the parameter storage information, thereby obtaining valid request parameters that have been batch requested for data within the predetermined time period; by requesting cached data according to the batch request parameters and detecting whether the return result of the cached data is empty, it is possible to detect whether the data cache contains cached data corresponding to the batch request parameters, and to perform corresponding processing according to the detection result; if the return result of the cached data is not empty, the update information of the cached data is obtained, and the cached data is updated according to the update information, thereby obtaining the request data after the cached data and the database are synchronized and refreshed; if the return result of the cached data is empty, the valid request parameters are filtered according to the update information to obtain the request parameters that have not been updated within the predetermined time period, and the request parameters are sent to the database to request data, thereby obtaining the request data within the predetermined time period at once, reducing the number of times the request parameters are accessed in the database. The above method can effectively improve the efficiency of batch data requests. This embodiment can be applied to product sales platforms. When a product sales platform needs to read data in batches, it can effectively improve the efficiency of data reading and further improve the data processing speed of the product sales platform. Attached Figure Description
[0055] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0056] Figure 1 This is an exemplary system architecture diagram to which this application can be applied;
[0057] Figure 2 A flowchart of an embodiment of the data bulk request method according to this application;
[0058] Figure 3 yes Figure 2 A flowchart of a specific implementation of step S10;
[0059] Figure 4 yes Figure 2 A flowchart of a specific implementation of step S20;
[0060] Figure 5 yes Figure 2A flowchart of a specific implementation of step S30;
[0061] Figure 6 yes Figure 2 A flowchart of a specific implementation of step S40;
[0062] Figure 7 yes Figure 2 A flowchart of a specific implementation of step S50;
[0063] Figure 8 This is a schematic diagram of a structure of an embodiment of the data bulk request apparatus according to this application;
[0064] Figure 9 This is a schematic diagram of the structure of one embodiment of the computer device according to this application. Detailed Implementation
[0065] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.
[0066] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a non-related or alternative embodiment that is mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.
[0067] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.
[0068] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.
[0069] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social media platform software, etc.
[0070] Terminal devices 101, 102, and 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 players (Moving Picture Experts Group Audio Layer IV), laptops, and desktop computers, etc.
[0071] Server 105 can be a server that provides various services, such as a backend server that supports the pages displayed on terminal devices 101, 102, and 103.
[0072] It should be noted that the data batch request method provided in this application embodiment is generally executed by the server, and correspondingly, the data batch request device is generally set in the server.
[0073] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.
[0074] Continue to refer to Figure 2 A flowchart illustrating an embodiment of a method for bulk data request according to this application is shown. The bulk data request method includes the following steps:
[0075] Step S10: Obtain a batch request instruction and obtain the corresponding batch request parameters according to the batch request instruction;
[0076] In this embodiment, a batch request instruction refers to a set of request instructions that contain multiple request instructions of the same type and are sent at once to obtain batch request data. Since a batch request instruction contains multiple request instructions, although the request data corresponding to these request instructions are of the same type, the content of the request parameters is different. Therefore, when obtaining the request parameters of the request data, it is necessary to first split the batch request instruction, obtain the request parameters corresponding to each request instruction separately, and then combine the individual request parameters into a request parameter set. When the content of the request parameter set corresponds to the batch request instruction, the request parameter set is used as the batch request parameter.
[0077] Step S20: Obtain preset parameter saving information, and save the batch request parameters within a predetermined time period according to the parameter saving information to obtain valid request parameters;
[0078] In this embodiment, the purpose of setting parameter saving information is to save the set of batch request parameters within a predetermined time period for convenient subsequent unified processing. The predetermined time period corresponding to the parameter saving information can be set according to the actual situation. In this embodiment, the predetermined time period is the time period from the moment when the batch request data begins to the moment one minute before the moment when the batch request data begins to be requested. For example, if the moment when the batch request data begins to be requested is 2:50:30, then the predetermined time period is 2:49:30.
[0079] Step S30: Request cached data according to the batch request parameters, and check whether the returned result of the cached data is empty;
[0080] In this embodiment, an empty cached data return result means that the batch data request could not find the cached data, or the cached data returned result was either unresponsive or empty, indicating that the cached data does not exist. Similarly, a non-empty cached data return result means that the batch data request could find the cached data and successfully returned the cached data.
[0081] Step S40: If the returned result of the cached data is not empty, then obtain the update information of the cached data, and update the cached data according to the update information to obtain the batch request data; and
[0082] In this embodiment, the update information is mainly used to update the cached data to effectively ensure that the cached data and the corresponding data stored in the database are synchronized and consistent.
[0083] Step S50: If the return result of the cached data is empty, the valid request parameters are filtered according to the update information to obtain the request parameters, and the request parameters are sent to the database to request data to obtain the batch request data.
[0084] In this embodiment, valid request parameters are filtered by updating information to effectively obtain data that has not been updated within a predetermined time period. This data includes cached data that failed to update during cached data updates and data for which no cached data was found, i.e., the cached data return result is empty. By filtering valid request parameters to obtain the request parameters corresponding to the above data, these parameters are used as the parameters to be requested for a unified data request, thereby obtaining these request data at once. These data are then stored in the data cache area in a predetermined order, so that the cached data in the data cache area can be directly obtained when requesting these data again.
[0085] This embodiment obtains batch request instructions and corresponding batch request parameters based on these instructions, facilitating subsequent queries of cached data. It also obtains preset parameter storage information and saves the batch request parameters for a predetermined time period, thus acquiring valid request parameters that have been used for batch requests within that time period. By requesting cached data based on the batch request parameters and checking if the cached data return result is empty, it detects whether the data cache contains cached data corresponding to the batch request parameters, and performs corresponding processing based on the detection result. If the cached data return result is not empty, it obtains update information for the cached data and updates it accordingly, resulting in request data synchronized with the database. If the cached data return result is empty, it filters valid request parameters based on the update information, obtaining pending request parameters that have not been updated within the predetermined time period. These pending request parameters are then sent to the database to request data, thus obtaining the request data for the predetermined time period in one go, reducing the number of database accesses for request parameters. This method effectively improves the efficiency of batch data requests. This embodiment can be applied to product sales platforms. When a product sales platform needs to read data in batches, it can effectively improve the efficiency of data reading and further improve the data processing speed of the product sales platform.
[0086] In this embodiment, the above method can be applied to a product marketing platform to make batch requests for data. When making a batch request, the corresponding batch request parameters are first obtained. These batch request parameters correspond to the attributes of the product marketing data, including the name, recording time, and data type of the product marketing data. By saving the batch request parameters within a predetermined time period and detecting whether product marketing data is cached in the batch request parameter request data cache area, the batch request parameters can be processed according to different situations, thereby effectively obtaining the product marketing data corresponding to the batch request parameters. For example, if the request parameters are the sneaker sales data name, sales time, and sales data type (text), the system first checks if the request is made within a predetermined time period. If it is, the request parameters are saved. Simultaneously, the system searches the data cache for cached sneaker sales data using the sneaker sales data name, sales time, and sales data type (text). If the corresponding sneaker sales data is found, it checks if the data has been updated. If not, it automatically retrieves the corresponding data from the database and updates it, ensuring real-time synchronization of the sneaker sales data. The updated sneaker sales data is then directly returned to the requesting client. If the corresponding sneaker sales data is not found, the saved request parameters are retrieved, and the request is sent to the database. The sneaker sales data is then directly returned to the requesting client, and the requested sneaker sales data is asynchronously saved to the data cache. This achieves fast and efficient retrieval of sneaker sales data.
[0087] Continue to refer to Figure 3 In some optional implementations of this embodiment, step S10 includes the following steps:
[0088] Step S101: Monitor the requesting end, obtain the batch request operation of the requesting end, and generate the batch request instruction according to the batch request operation;
[0089] In this embodiment, data tracking is implemented on the requesting end to achieve effective monitoring. When the requesting end detects that a request operation has occurred, it will trigger the corresponding event for recording and generate the corresponding request instruction based on the event, thereby generating batch request data based on batch request operations.
[0090] Step S102: The batch request instruction is split into separate instruction segments to obtain the request instruction; and
[0091] In this embodiment, since the batch request instruction contains multiple data request instructions, the request parameters corresponding to the batch request instruction cannot be obtained all at once. It is necessary to split the batch request instruction into individual request instructions in order to effectively obtain the request parameters corresponding to each request instruction.
[0092] Step S103: Obtain the corresponding request parameters according to the request instruction, and merge and organize the request parameters to obtain the batch request parameters.
[0093] In this embodiment, merging and organizing request parameters means organizing individual request parameters into a request parameter set, and using this request parameter set as batch request parameters.
[0094] Continue to refer to Figure 4 In some optional implementations of this embodiment, step S20 includes the following steps:
[0095] Step S201: Obtain the storage time information and parse the storage time information to obtain the storage duration;
[0096] In this embodiment, the storage duration refers to the effective duration of storing batch request parameters. By setting the storage duration, the storage time of batch request parameters can be set, so that the effective request parameters always have a certain timeliness.
[0097] Step S202: Obtain the request time of the batch request parameters, and sort the request times to obtain a request time sorting table;
[0098] In this embodiment, the request time sorting table is sorted from newest to oldest time to facilitate subsequent querying and retrieval of request times.
[0099] Step S203: The request time closest to the current time in the request time sorting table is used as the starting time for saving;
[0100] In this embodiment, since the request time sorting table is sorted from newest to oldest, the request time most recent to the current time should be the first position in the request time sorting table. Therefore, the first position of the request time sorting table can be directly obtained as the starting time for saving. To ensure the validity of the request time sorting table, it should be updated at predetermined time intervals. In this embodiment, the predetermined time interval is set to 1 second to ensure effective updates. This predetermined time interval can be adjusted according to actual circumstances.
[0101] Step S204: Calculate the predetermined time period based on the storage start time and storage duration;
[0102] In this embodiment, the start time and duration of the data are calculated by subtracting the duration from the start time. For example, if the start time is 2:30 and the duration is 1 minute, the predetermined time period is from 2:29 to 2:30. The duration can be adjusted according to actual circumstances.
[0103] Step S205: Obtain the request parameters to be saved corresponding to the predetermined time period from the request time sorting table, and asynchronously save the request parameters to be saved according to the saving method information to obtain the save request parameters;
[0104] In this embodiment, by using asynchronous saving, the system can save the request parameters that meet the conditions without affecting the batch request parameters, thus effectively ensuring the data query speed of the system.
[0105] Step S206: Detect whether the request parameters to be saved have been successfully saved based on the save check information;
[0106] In this embodiment, since the request parameters to be saved are saved asynchronously, in order to ensure that the request parameters recorded in the valid request parameters are valid, it is necessary to detect whether the request parameters to be saved have been saved successfully, so that if the request data to be saved fails to be saved, the request data to be saved is saved again.
[0107] Step S207: If the request parameters to be saved are saved successfully, then the saved request parameters are used as the valid request parameters; and
[0108] Step S208: If the request parameter to be saved fails to be saved, the request parameter to be saved is retrieved again and saved again until the request parameter to be saved is saved successfully.
[0109] In this embodiment, since the request parameter to be saved has a certain time sensitivity, if the time for repeatedly saving the request parameter reaches a preset saving threshold, the saving action will stop to release the system's running memory. In this embodiment, the saving threshold is preset to 5 times. If the number of times the request parameter to be saved is repeatedly saved reaches 5 times, the saving action for the request parameter to be saved will stop.
[0110] Continue to refer to Figure 5 In some optional implementations of this embodiment, step S30 includes the following steps:
[0111] Step S301: Obtain the batch request parameters and send the batch request parameters to the data cache area to query the cached data;
[0112] In this embodiment, SQL statements can be used to query the cached data corresponding to the batch request parameters in the data cache area.
[0113] Step S302: Extract the cached data and return the cached data to the requesting end to obtain the return result; and
[0114] Step S303: Obtain preset null value detection information, and detect whether the returned result is null based on the control detection information.
[0115] In this embodiment, isnull() is used to detect whether the returned result is empty.
[0116] Continue to refer to Figure 6 In some optional implementations of this embodiment, step S40 includes the following steps:
[0117] Step S401: Obtain the preset update duration and the request time of the batch request data, and calculate the update period corresponding to the cached data based on the request time and the update duration;
[0118] In this embodiment, in order to ensure the real-time and effectiveness of data updates, an update duration needs to be set to ensure the timeliness of the update. The update period is calculated as the time between the request time minus the update duration and the request time. For example, if the request time is 2:30 and the update duration is 3 minutes, then the update period is from 2:27 to 2:30.
[0119] Step S402: Monitor the data cache area corresponding to the cached data, and record the operation behavior of the data cache area during the update period, wherein the operation behavior includes at least update behavior;
[0120] In this embodiment, the operation behavior also includes query behavior, write behavior, and read behavior. In this embodiment, only the update behavior among the operation behaviors is acquired. The acquisition method can be achieved by monitoring whether an update event corresponding to the update behavior is generated in the data cache area.
[0121] Step S403: Associate the operation behavior with the cached data, and generate the update information based on the operation behavior;
[0122] Step S404: Detect whether the update information includes the update behavior;
[0123] Step S405: If the update information includes the update behavior, then the cached data is used as the batch request data; and
[0124] Step S406: If the update information does not include the update behavior, then obtain the data identifier of the cached data, search for the corresponding update cached data in the database according to the data identifier, and update the cached data according to the update cached data.
[0125] In this embodiment, the data identifier of the cached data can be the data ID information. The corresponding updated cached data can be effectively found in the database through the data identifier so as to update the cached data.
[0126] Continue to refer to Figure 7 In some optional implementations of this embodiment, step S50 includes the following steps:
[0127] Step S501: Obtain the update information and determine the corresponding update request parameters based on the update information;
[0128] Step S502: Obtain the valid request parameters; filter the valid request parameters according to the update request parameters to obtain the parameters to be requested; and
[0129] In this embodiment, the filtering operation is achieved by identifying the request parameters corresponding to the update request parameters among the valid request parameters and removing those parameters accordingly.
[0130] Step S503: Send the requested parameters to the database to query the batch request data, and extract the batch request data.
[0131] In some optional implementations of this embodiment, the following steps are included before step S503:
[0132] Obtain error information from the database and detect whether the error information reaches a preset threshold;
[0133] In this embodiment, the error message refers to the error message that appears in the database program window. When an error message appears, it indicates that the database has been attacked or there is risky data. Therefore, in order to investigate the cause of the error message, it is necessary to close the database data interface.
[0134] If the error message reaches the preset threshold, the database data interface will be changed from an open state to a closed state, and empty data will be returned when the requested parameters are used for data querying; and
[0135] In this embodiment, to avoid false error reports from the database, a preset threshold is set to detect whether false error reports are occurring. When the number of consecutive errors from the database reaches the preset threshold, it is confirmed that the database is experiencing a genuine error. At this point, the database's data interface is closed to facilitate investigation of the cause of the error. In this embodiment, the preset threshold is 3 times, which can be adjusted according to actual circumstances.
[0136] If the error message does not reach the preset threshold, the data interface remains open until the database error message reaches the preset threshold or the database closes the data interface.
[0137] In this embodiment, the database closes the data interface by actively closing it upon receiving an instruction.
[0138] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware with computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium. When executed, the program can include the processes of the embodiments of the above methods. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).
[0139] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.
[0140] Further reference Figure 8 As a response to the above Figure 1 To implement the method shown, this application provides an embodiment of a data batch request device, which is similar to... Figure 1 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.
[0141] like Figure 8 As shown, the data batch request device 600 described in this embodiment includes: a parameter acquisition module 601, a parameter storage module 602, a data detection module 603, a first data request module 604, and a second data request module 605. Wherein:
[0142] The parameter acquisition module 601 is used to acquire a batch request instruction and acquire the corresponding batch request parameters according to the batch request instruction.
[0143] The parameter saving module 602 is used to obtain preset parameter saving information, and save the batch request parameters within a predetermined time period according to the parameter saving information to obtain valid request parameters;
[0144] The data detection module 603 is used to request cached data according to the batch request parameters and detect whether the returned result of the cached data is empty;
[0145] The first data request module 604 is configured to, if the return result of the cached data is not empty, obtain the update information of the cached data, and update the cached data according to the update information to obtain batch request data; and
[0146] The second data request module 605 is used to filter the valid request parameters according to the update information if the return result of the cached data is empty, obtain the parameters to be requested, and send the parameters to be requested to the database to make a data request, thereby obtaining the batch request data.
[0147] This embodiment, by employing the above-described device, can effectively improve the efficiency of data reading when making batch requests for data.
[0148] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 9 , Figure 9 This is a basic structural block diagram of the computer device in this embodiment.
[0149] The computer device 7 includes a memory 71, a processor 72, and a network interface 73 that are interconnected via a system bus. It should be noted that only the computer device 7 with components 71-73 is shown in the figure; however, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described here is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.
[0150] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.
[0151] The memory 71 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 71 may be an internal storage unit of the computer device 7, such as the hard disk or memory of the computer device 7. In other embodiments, the memory 71 may also be an external storage device of the computer device 7, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 7. Of course, the memory 71 may include both the internal storage unit and its external storage device of the computer device 7. In this embodiment, the memory 71 is typically used to store the operating system and various application software installed on the computer device 7, such as computer-readable instructions for data batch request methods. In addition, the memory 71 can also be used to temporarily store various types of data that have been output or will be output.
[0152] In some embodiments, the processor 72 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 72 is typically used to control the overall operation of the computer device 7. In this embodiment, the processor 72 is used to execute computer-readable instructions stored in the memory 71 or to process data, for example, to execute computer-readable instructions for the data bulk request method.
[0153] The network interface 73 may include a wireless network interface or a wired network interface, which is typically used to establish communication connections between the computer device 7 and other electronic devices.
[0154] This embodiment, by employing the aforementioned computer equipment, can effectively improve the efficiency of data reading when making batch requests for data.
[0155] This application also provides another embodiment, namely, providing a computer-readable storage medium storing computer-readable instructions that can be executed by at least one processor to cause the at least one processor to perform the steps of the data bulk request method described above.
[0156] This embodiment, by employing the aforementioned computer-readable storage medium, can effectively improve the efficiency of data reading when making batch requests for data.
[0157] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.
[0158] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, the purpose of providing these embodiments is to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application.
Claims
1. A method for batch data requests, characterized in that, Includes the following steps: Obtain a batch request instruction, and obtain the corresponding batch request parameters based on the batch request instruction; Obtain preset parameter saving information, and save the batch request parameters within a predetermined time period according to the parameter saving information to obtain valid request parameters; Request cached data according to the batch request parameters, and check whether the returned result of the cached data is empty; If the returned result of the cached data is not empty, then obtain the update information of the cached data, and update the cached data according to the update information to obtain the batch request data; and If the cached data returns an empty result, the valid request parameters are filtered according to the update information to obtain the parameters to be requested, and the parameters to be requested are sent to the database to request data to obtain the batch request data; The parameter storage information includes storage time information, storage method information, and storage check information. The step of obtaining preset parameter storage information, and storing the batch request parameters within a predetermined time period based on the parameter storage information to obtain valid request parameters specifically includes: Obtain the storage time information and parse the storage time information to obtain the storage duration; Obtain the request time of the batch request parameters, and sort the request times to obtain a request time sorting table; The request time closest to the current time in the request time sorting table is used as the starting time for saving; The predetermined time period is calculated based on the storage start time and storage duration; Obtain the request parameters to be saved corresponding to the predetermined time period from the request time sorting table, and asynchronously save the request parameters to be saved according to the saving method information to obtain the saved request parameters; The save check information is used to determine whether the request parameters to be saved were saved successfully. If the request parameters to be saved are saved successfully, then the saved request parameters will be used as the valid request parameters; and If the request parameters to be saved fail to be saved, the request parameters to be saved are retrieved again and saved again until the request parameters to be saved are saved successfully. Specifically, the request time of the first position in the request time sorting table is used as the starting time for saving, and the request time sorting table is updated at predetermined time intervals. The step of obtaining update information for the cached data and updating the cached data according to the update information to obtain batch request data specifically includes: Obtain the preset update duration and the request time of the batch request data, and calculate the update period corresponding to the cached data based on the request time and the update duration; Monitor the data cache area corresponding to the cached data, and record the operation behavior of the data cache area during the update period, wherein the operation behavior includes at least update behavior; The operation behavior is associated with the cached data, and the update information is generated based on the operation behavior; Detect whether the update information includes the update behavior; If the update information includes the update behavior, then the cached data will be used as the batch request data; and If the update information does not include the update behavior, then obtain the data identifier of the cached data, search for the corresponding updated cached data in the database according to the data identifier, and update the cached data according to the updated cached data; The update period is calculated as the time between the request time minus the update duration and the request time.
2. The data batch request method according to claim 1, characterized in that, The step of obtaining a batch request instruction and obtaining the corresponding batch request parameters based on the batch request instruction specifically includes: Monitor the requesting end, obtain the batch request operations of the requesting end, and generate the batch request instruction based on the batch request operations; The batch request instruction is split into separate instruction segments to obtain the request instruction; and The corresponding request parameters are obtained according to the request instruction, and the request parameters are merged and organized to obtain the batch request parameters.
3. The data batch request method according to claim 1, characterized in that, The step of requesting cached data according to the batch request parameters and detecting whether the returned result of the cached data is empty specifically includes: Obtain the batch request parameters and send the batch request parameters to the data cache to query the cached data; The cached data is extracted and returned to the requesting end to obtain the return result; and Obtain preset null value detection information, and detect whether the returned result is null based on the null value detection information.
4. The data batch request method according to claim 1, characterized in that, The step of filtering the valid request parameters according to the updated information to obtain the parameters to be requested, and sending the parameters to be requested to the database to request data, thereby obtaining the batch request data, specifically includes: Obtain the update information and determine the corresponding update request parameters based on the update information; Obtain the valid request parameters, filter the valid request parameters according to the update request parameters, and obtain the parameters to be requested; and The requested parameters are sent to the database to query the batch request data, and the batch request data is extracted.
5. The data batch request method according to claim 1, characterized in that, Before the steps of sending the requested parameters to the database to query the batch request data and extracting the batch request data, the following steps are also included: Obtain error information from the database and detect whether the error information reaches a preset threshold; If the error message reaches the preset threshold, the database data interface will be changed from an open state to a closed state, and empty data will be returned when the requested parameters are used for data querying; and If the error message does not reach the preset threshold, the data interface remains open until the database error message reaches the preset threshold or the database closes the data interface.
6. A data batch request device, characterized in that, include: The parameter acquisition module is used to acquire batch request instructions and obtain corresponding batch request parameters based on the batch request instructions. The parameter saving module is used to obtain preset parameter saving information, and save the batch request parameters within a predetermined time period according to the parameter saving information to obtain valid request parameters; The data detection module is used to request cached data according to the batch request parameters and to detect whether the returned result of the cached data is empty. The first data request module is used to obtain the update information of the cached data if the return result of the cached data is not empty, and update the cached data according to the update information to obtain batch request data; and The second data request module is used to filter the valid request parameters according to the update information if the return result of the cached data is empty, obtain the parameters to be requested, and send the parameters to be requested to the database to make a data request, thereby obtaining the batch request data; The parameter storage information includes storage time information, storage method information, and storage check information. Specifically, the parameter storage module is used to: acquire the storage time information and parse it to obtain a storage duration; acquire the request time of the batch request parameters and sort the request times to obtain a request time sorting table; use the request time closest to the current time in the request time sorting table as the storage start time; calculate the predetermined time period based on the storage start time and storage duration; acquire the request parameters to be saved corresponding to the predetermined time period from the request time sorting table and asynchronously save the request parameters to be saved according to the storage method information to obtain the storage request parameters; check whether the request parameters to be saved have been saved successfully according to the storage check information; if the request parameters to be saved successfully, then use the storage request parameters as the valid request parameters; and if the request parameters to be saved unsuccessfully, then reacquire the request parameters to be saved and resave them until the request parameters to be saved successfully. Specifically, the request time of the first position in the request time sorting table is used as the starting time for saving, and the request time sorting table is updated at predetermined time intervals. The process includes: obtaining a preset update duration and the request time of the batch request data; calculating the update period corresponding to the cached data based on the request time and the update duration; monitoring the data cache area corresponding to the cached data and recording the operation behavior of the data cache area within the update period, wherein the operation behavior includes at least an update behavior; associating the operation behavior with the cached data and generating update information based on the operation behavior; detecting whether the update information includes the update behavior; if the update information includes the update behavior, then using the cached data as the batch request data; and if the update information does not include the update behavior, then obtaining the data identifier of the cached data, searching for the corresponding updated cached data in the database based on the data identifier, and updating the cached data based on the updated cached data. The update period is calculated as the time between the request time minus the update duration and the request time.
7. A computer device, characterized in that, The device includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the data bulk request method as described in any one of claims 1 to 5.
8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions that, when executed by a processor, implement the steps of the data bulk request method as described in any one of claims 1 to 5.