A method for batch processing of massive data and electronic equipment

Abstract

The present application discloses a method and electronic device for batch processing of massive data, belonging to the field of computer technology. The method includes: uploading a batch file to be queried, initiating a query request; determining the type of the query request, wherein the type of the query request includes batch query and batch skip query; and performing data query according to the type of the query request. The present application can support batch query of large amounts of data, quickly retrieve and respond to users, and improve user experience.

Description

Technical Field

[0001] The present application belongs to the field of computer technology, and in particular relates to a method and electronic device for batch processing of massive data. Background Art

[0002] Among the technologies related to database management, the implementation of batch query technology usually involves multiple levels such as front-end application, middleware, database management system, etc. Among them, the front-end application is responsible for receiving user requests and displaying results; the middleware is responsible for processing requests and converting them into query statements that can be recognized by the database; the database management system is responsible for executing queries and returning results.

[0003] When the amount of data is very large, batch query faces the problem of low query efficiency, which will cause users to wait too long and respond in timeout, affecting user experience.

[0004] In view of the above problems, a method and electronic device for batch processing of massive data are proposed in the present application. Summary of the invention

[0005] In order to address the deficiencies of the prior art, the present application provides a method and electronic device for batch processing of massive data, which can solve the problem of low query efficiency in batch queries, resulting in long user waiting time, response timeouts, and affected user experience.

[0006] The technical effects to be achieved by this application are achieved through the following solutions:

[0007] In a first aspect, the present application provides a method for batch processing of massive data, the method comprising:

[0008] Upload the batch files to be queried and initiate a query request;

[0009] Determine the type of the query request, wherein the type of the query request includes batch query and skip batch query;

[0010] Data query is performed according to the type of the query request.

[0011] In some embodiments, when the type of the query request is a batch query, performing the data query includes:

[0012] Obtaining a first query corresponding to the current batch, and if all query results corresponding to the first query exist in the cache, directly obtaining all query results corresponding to the first query from the cache, wherein the first query represents a query for at least one set of data;

[0013] If all query results corresponding to the first query do not exist in the cache, then determine in turn whether the query result of each group of data corresponding to the first query exists in the cache; if the query result of the group of data exists in the cache, obtain the query result of the group of data from the cache and increase the cache weight value corresponding to the group of data by 1; if the query result of the group of data does not exist in the cache, obtain the query result corresponding to the group of data from the database, store the query result corresponding to the group of data in the cache, and continue to query the next group of data until the sum of the number of all queried groups is greater than the first batch threshold of the current batch, then assemble and return the queried query results, and store the complete query results corresponding to the current batch in the cache.

[0014] In some embodiments, when the type of the query request is a skip batch query, performing the data query includes:

[0015] Obtaining a second query corresponding to the current batch, and if all query results corresponding to the second query exist in the cache, directly obtaining all query results corresponding to the second query from the cache, wherein the second query represents a query for multiple groups;

[0016] If all query results corresponding to the second query do not exist in the cache, determine the correspondence between each group of data and the quantity, determine the target interval of the query based on the second batch threshold of the current batch and the correspondence, determine at least one group of data for the query according to the target interval, and perform data query.

[0017] In some embodiments, after the first query of the current batch is completed, the next batch of asynchronous queries is started.

[0018] In some embodiments, determining the correspondence between each set of data and the quantity includes:

[0019] Determine whether there is a corresponding relationship between each group of data and the quantity in the cache according to the batch file; if the corresponding relationship exists in the cache, obtain the corresponding relationship from the cache; if the corresponding relationship does not exist in the cache, perform a quantity query for each group of data, determine the corresponding relationship between each group of data and the quantity, and store the corresponding relationship in the cache.

[0020] In some embodiments, determining the target interval of the query based on the second batch threshold of the current batch and the corresponding relationship includes:

[0021] Obtaining the endpoint group data corresponding to the previous batch, wherein the endpoint group data represents the last group of data that has been queried in the previous batch;

[0022] Starting from the next group of data of the endpoint group data, the total number of N groups of data is calculated in sequence. When the total number is greater than the second batch threshold corresponding to the current batch, it is determined that the target interval of the current batch is from the endpoint group data and the N groups of data, where N is an integer greater than or equal to 1.

[0023] In some embodiments, determining at least one set of data to be queried according to the target interval and performing the data query includes:

[0024] It is determined in turn whether the query result of each group of data in the at least one group of data exists in the cache; if the query result of the group of data exists in the cache, the query result of the group of data is obtained from the cache and the cache weight value corresponding to the group of data is increased by 1; if the query result of the group of data does not exist in the cache, the query result corresponding to the group of data is obtained from the database, and the query result corresponding to the group of data is stored in the cache, and the next group of data is continued to be queried until the sum of the number of all queried groups is greater than the second batch threshold of the current batch, then the queried query results are assembled and returned, and the complete query results corresponding to the current batch are stored in the cache.

[0025] In some embodiments, the method further comprises:

[0026] Determine whether the cache weight value of each group of data in the cache is greater than the weight threshold. If it is greater than the weight threshold, continue to retain the corresponding cache data; if it is less than or equal to the weight threshold, delete the corresponding cache data.

[0027] In a second aspect, the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any one of the methods described above when executing the computer program.

[0028] In a third aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement any of the methods described above.

[0029] The batch processing method for massive data provided by the embodiment of the present application determines the type of query request, and adopts different query strategies to query data according to batch query or skip batch query. It can support batch query of large data volumes, quickly retrieve and respond to users, and improve user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present application or the existing technical solutions, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0031] Figure 1 This is a flow chart of a method for batch processing of massive data in one embodiment of the present application;

[0032] Figure 2 It is a schematic block diagram of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0034] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in one or more embodiments of the present application should be understood by people with ordinary skills in the field to which the present application belongs. The "first", "second" and similar words used in one or more embodiments of the present application do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0035] Various non-limiting implementations of the present application are described in detail below in conjunction with the accompanying drawings.

[0036] First, refer to Figure 1 , the batch processing method of the massive data of this application is described in detail:

[0037] The present application provides a method for batch processing of massive data, the method comprising:

[0038] S1: Upload the batch files to be queried and initiate a query request;

[0039] S2: Determine the type of the query request, where the type of the query request includes batch query and skip batch query;

[0040] S3: Perform data query according to the type of the query request.

[0041] In some embodiments, when the type of the query request is a batch query, performing the data query includes:

[0042] Obtaining a first query corresponding to the current batch, and if all query results corresponding to the first query exist in the cache, directly obtaining all query results corresponding to the first query from the cache, wherein the first query represents a query for at least one set of data;

[0043] Specifically, the first query represents an overall query of the current batch to check whether there are query results corresponding to all the group data of the current batch in the cache. If all the query results corresponding to the current batch are in the cache, there is no need to execute queries by group data in sequence, which can improve efficiency and reduce costs.

[0044] If all query results corresponding to the first query do not exist in the cache, then determine in turn whether the query results of each group of data corresponding to the first query exist in the cache. If the query results of the group of data exist in the cache, obtain the query results of the group of data from the cache and add 1 to the cache weight value corresponding to the group of data. If the query results of the group of data do not exist in the cache, obtain the query results corresponding to the group of data from the database, and store the query results corresponding to the group of data in the cache. Continue to query the next group of data until the sum of the number of all groups queried is greater than the first batch threshold of the current batch. Then assemble and return the query results obtained, and store the complete query results corresponding to the current batch in the cache. The complete query results corresponding to the current batch (i.e., the query results corresponding to all groups of data) are uniformly stored in the cache so that the overall query by batch can be performed later. This can improve query efficiency and user experience.

[0045] Specifically, each batch has a corresponding batch threshold. The first batch threshold mentioned above indicates the batch threshold corresponding to the current batch, for example, the batch threshold is 25, where "first" and "second" are only used for distinction and have no special meaning; the second batch threshold mentioned below indicates the batch threshold corresponding to the current batch in the skip batch query, that is, the first batch threshold and the second batch threshold in this article are only used to distinguish different thresholds, and the sizes of different thresholds can be the same or different, without specific limitation.

[0046] In some embodiments, after the first query of the current batch is completed, the next batch of asynchronous queries is started.

[0047] Specifically, first determine whether each group of data already has cached data. If the query result of this group is already in the cache, then directly take the result from the cache and skip this group of data to query the next group of data (at the same time increase the cache weight of this group of data by +1); if there is no result in the cache, scan and query in the database; when the sum of the query results meets the next batch threshold, stop the subsequent scan, and then organize and store the query results corresponding to each group of data found in the cache. If the sum of the query results meets the next batch threshold, the asynchronous query ends.

[0048] Specifically, the above-mentioned sum of query results meeting the next batch threshold means that the sum of the data amounts of the query results is greater than the next batch threshold.

[0049] For example, the batch query can be performed as follows:

[0050] 1. Upload the batch files to be queried, such as 100 sets of data, and initiate a query request.

[0051] 2. Determine the type of query request. If it is a batch query, assume that the current query batch is the first batch, and the first batch threshold is 25, and the next batch threshold is also 25.

[0052] 3. Obtain the first query corresponding to the first batch, the first query corresponds to the query of at least one set of data, and query each set of data corresponding to the first query separately.

[0053] 4. First, obtain the query results corresponding to the first batch from the cache. If all the corresponding results exist in the cache, directly obtain all the query results from the cache and feed back the results to the user. If all the corresponding query results do not exist in the cache, determine in turn whether each group of data in at least one group of data exists in the cache. Each group of data existing in the cache is obtained from the cache, and each group of data not existing in the cache is obtained from the database. The query results obtained from the database that do not exist in the cache are stored in the corresponding cache. For the corresponding query results directly obtained from the cache, the cache weight value corresponding to 1 is increased, indicating that the query result is data with a higher query frequency and needs to be saved for a longer time.

[0054] 5. Assume that 10 query results are found in the first set of data. Since 10<25, the second set of data needs to be checked. If 20 query results are obtained in the second set, (10+20)>25. Subsequent scanning is stopped and the 25 matching data are sorted and returned.

[0055] 6. Then asynchronously query the next batch of data. The specific query steps are the same as the query method for the first batch, so they will not be repeated here.

[0056] In some embodiments, when the type of the query request is a skip batch query, performing the data query includes:

[0057] Obtaining a second query corresponding to the current batch, and if all query results corresponding to the second query exist in the cache, directly obtaining all query results corresponding to the second query from the cache, wherein the second query represents a query for multiple groups;

[0058] If all query results corresponding to the second query do not exist in the cache, determine the correspondence between each group of data and the quantity, determine the target interval of the query based on the second batch threshold of the current batch and the correspondence, determine at least one group of data for the query according to the target interval, and perform data query.

[0059] In some embodiments, determining the correspondence between each set of data and the quantity includes:

[0060] Determine whether there is a corresponding relationship between each group of data and the quantity in the cache according to the batch file; if the corresponding relationship exists in the cache, obtain the corresponding relationship from the cache; if the corresponding relationship does not exist in the cache, perform a quantity query for each group of data, determine the corresponding relationship between each group of data and the quantity, and store the corresponding relationship in the cache.

[0061] In some embodiments, determining the target interval of the query based on the second batch threshold of the current batch and the corresponding relationship includes:

[0062] Obtaining the endpoint group data corresponding to the previous batch, wherein the endpoint group data represents the last group of data that has been queried in the previous batch;

[0063] Starting from the next group of data of the endpoint group data, the total number of N groups of data is calculated in sequence. When the total number is greater than the second batch threshold corresponding to the current batch, it is determined that the target interval of the current batch is from the endpoint group data and the N groups of data, where N is an integer greater than or equal to 1.

[0064] In some embodiments, determining at least one set of data to be queried according to the target interval and performing the data query includes:

[0065] It is determined in turn whether the query result of each group of data in the at least one group of data exists in the cache. If the query result of the group of data exists in the cache, the query result of the group of data is obtained from the cache and the cache weight value corresponding to the group of data is increased by 1. If the query result of the group of data does not exist in the cache, the query result corresponding to the group of data is obtained from the database, and the query result corresponding to the group of data is stored in the cache. The next group of data is continued to be queried until the sum of the number of all queried groups is greater than the second batch threshold of the current batch. The query results obtained are assembled and returned, and the complete query results corresponding to the current batch are stored in the cache, so that the overall query by batch can be performed later. This can improve query efficiency and enhance user experience.

[0066] For example, the batch skipping query may be performed as follows:

[0067] Skip batch query:

[0068] 1. Upload the batch files to be queried, such as 100 sets of data, and initiate a query request.

[0069] 2. Determine the type of query request. If it is a skip batch query, assume that the current query batch is the third batch. There are the first batch and the second batch before the third batch. The threshold of the first batch is 25, the threshold of the second batch is also 25, and the threshold of the third batch is also 25 (i.e., the threshold of the second batch). If the corresponding query result exists in the cache, directly obtain the query result. If the query result of the current query batch (i.e., the third batch) does not exist in the cache, continue with the subsequent steps.

[0070] 3. Determine the corresponding relationship between each group of data and quantity. For example, the corresponding relationship between group number and quantity is as follows:

[0071] 1----10

[0072] 2----5

[0073] 3-10

[0074] 4-15

[0075] 5----20

[0076] 6-10

[0077] 7-20

[0078] 8-15

[0079] …(the subsequent correspondence between group numbers and quantities is omitted).

[0080] 4. Determine the target interval of the query based on the batch threshold of the current batch (i.e. 25) and the corresponding relationship, where the calculation method of the second batch of the query is: (10+5+10+15+20)>2*25, that is, the second batch of data ends at the fifth group of data. Therefore, starting from the fifth group of data, the data interval of the third batch (i.e. the current batch) is: (20+10)>25, that is, the fifth and sixth groups of data can meet the query requirements of the third batch.

[0081] 5. Perform cache query and database query for each set of data that meets the requirements of the third batch. The specific query principles have been described in detail above and will not be repeated here.

[0082] In some embodiments, the method further comprises:

[0083] Determine whether the cache weight value of each group of data in the cache is greater than the weight threshold. If it is greater than the weight threshold, continue to retain the corresponding cache data. If it is less than or equal to the weight threshold, delete the corresponding cache data. This can improve the hit rate of data queries in the cache and improve efficiency.

[0084] The batch processing method for massive data provided by the embodiment of the present application determines the type of query request, and adopts different query strategies to query data according to batch query or skip batch query. It can support batch query of large data volumes, quickly retrieve and respond to users, and improve user experience.

[0085] It should be noted that the method of one or more embodiments of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of one or more embodiments of the present application, and the multiple devices will interact with each other to complete the described method.

[0086] It should be noted that the above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0087] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present application also discloses an electronic device;

[0088] Specifically, Figure 2 The hardware structure diagram of an electronic device for a method for batch processing of mass data provided in this embodiment is shown, and the device may include: a processor 410, a memory 420, an input / output interface 430, a communication interface 440, and a bus 450. The processor 410, the memory 420, the input / output interface 430, and the communication interface 440 are connected to each other in communication within the device through the bus 450.

[0089] The processor 410 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0090] The memory 420 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 420 may store an operating system and other application programs. When the technical solution provided in the embodiment of the present application is implemented by software or firmware, the relevant program code is stored in the memory 420 and is called and executed by the processor 410.

[0091] The input / output interface 430 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0092] The communication interface 440 is used to connect a communication module (not shown in the figure) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (for example, USB, network cable, etc.) or a wireless mode (for example, mobile network, WIFI, Bluetooth, etc.).

[0093] The bus 450 includes a path that transmits information between the various components of the device (eg, the processor 410 , the memory 420 , the input / output interface 430 , and the communication interface 440 ).

[0094] It should be noted that, although the above device only shows the processor 410, the memory 420, the input / output interface 430, the communication interface 440 and the bus 450, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiment of the present application, and does not necessarily include all the components shown in the figure.

[0095] The electronic device of the above embodiment is used to implement the corresponding batch processing method of massive data in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0096] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, one or more embodiments of the present application also provide a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the batch processing method of massive data as described in any of the above embodiments.

[0097] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0098] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the batch processing method of massive data as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0099] A person skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of different aspects of one or more embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0100] In addition, to simplify the description and discussion, and in order not to make one or more embodiments of the present application difficult to understand, the well-known power / ground connections to the integrated circuit (IC) chip and other components may or may not be shown in the provided figures. In addition, the device may be shown in the form of a block diagram to avoid making one or more embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform on which one or more embodiments of the present application will be implemented (that is, these details should be fully within the scope of understanding of those skilled in the art). Where specific details (e.g., circuits) are set forth to describe exemplary embodiments of the present application, it is obvious to those skilled in the art that one or more embodiments of the present application can be implemented without these specific details or with changes in these specific details. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0101] Although the present application has been described in conjunction with specific embodiments of the present application, many alternatives, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the discussed embodiments.

[0102] One or more embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of the present application should be included in the scope of protection of the present application.

Claims

1. A method for batch processing of massive data, characterized in that: The method comprises: Upload the batch files to be queried and initiate a query request; Determine the type of the query request, wherein the type of the query request includes batch query and skip batch query; Performing data query according to the type of the query request; When the query request type is a batch query, performing data query includes: Obtaining a first query corresponding to the current batch, and if all query results corresponding to the first query exist in the cache, directly obtaining all query results corresponding to the first query from the cache, wherein the first query represents a query for at least one set of data; In the case where the type of the query request is a skip batch query, performing the data query includes: Obtaining a second query corresponding to the current batch, and if all query results corresponding to the second query exist in the cache, directly obtaining all query results corresponding to the second query from the cache, wherein the second query represents a query for multiple groups; In the case where the type of the query request is batch query, the performing of data query further includes: If all query results corresponding to the first query do not exist in the cache, then determine in turn whether the query result of each group of data corresponding to the first query exists in the cache; if the query result of the group of data exists in the cache, obtain the query result of the group of data from the cache and add 1 to the cache weight value corresponding to the group of data; if the query result of the group of data does not exist in the cache, obtain the query result corresponding to the group of data from the database, store the query result corresponding to the group of data in the cache, and continue to query the next group of data until the sum of the number of all queried groups is greater than the first batch threshold of the current batch, then assemble and return the queried query results, and store the complete query results corresponding to the current batch in the cache; In the case where the type of the query request is a skip batch query, performing the data query further includes: If all query results corresponding to the second query do not exist in the cache, determine the correspondence between each group of data and the quantity, determine the target interval of the query based on the second batch threshold of the current batch and the correspondence, determine at least one group of data for the query according to the target interval, and perform data query.

2. The method for batch processing of mass data according to claim 1, characterized in that: After the first query of the current batch is completed, the next batch of asynchronous queries is started.

3. The method for batch processing of mass data according to claim 1, characterized in that: Determining the corresponding relationship between each set of data and the quantity includes: Determine whether there is a corresponding relationship between each group of data and the quantity in the cache according to the batch file; if the corresponding relationship exists in the cache, obtain the corresponding relationship from the cache; if the corresponding relationship does not exist in the cache, perform a quantity query for each group of data, determine the corresponding relationship between each group of data and the quantity, and store the corresponding relationship in the cache.

4. The method for batch processing of mass data according to claim 1, characterized in that: The determining the target interval of the query based on the second batch threshold of the current batch and the corresponding relationship includes: Obtaining the endpoint group data corresponding to the previous batch, wherein the endpoint group data represents the last group of data that has been queried in the previous batch; Starting from the next group of data of the endpoint group data, the total number of N groups of data is calculated in sequence. When the total number is greater than the second batch threshold corresponding to the current batch, it is determined that the target interval of the current batch is from the endpoint group data and the N groups of data, where N is an integer greater than or equal to 1.

5. The method for batch processing of mass data according to claim 4, characterized in that: The step of determining at least one set of data to be queried according to the target interval and performing data query includes: It is determined in turn whether the query result of each group of data in the at least one group of data exists in the cache; if the query result of the group of data exists in the cache, the query result of the group of data is obtained from the cache and the cache weight value corresponding to the group of data is increased by 1; if the query result of the group of data does not exist in the cache, the query result corresponding to the group of data is obtained from the database, and the query result corresponding to the group of data is stored in the cache, and the next group of data is continued to be queried until the sum of the number of all queried groups is greater than the second batch threshold of the current batch, then the queried query results are assembled and returned, and the complete query results corresponding to the current batch are stored in the cache.

6. The method for batch processing of mass data according to claim 1, characterized in that: The method further comprises: Determine whether the cache weight value of each group of data in the cache is greater than the weight threshold. If it is greater than the weight threshold, continue to retain the corresponding cache data; if it is less than or equal to the weight threshold, delete the corresponding cache data.

7. An electronic device, characterized in that: The electronic device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 6 when executing the computer program.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the method according to any one of claims 1 to 6.

Images