A data query method and device, electronic equipment and storage medium

By adopting a three-tier storage structure and query strategy in the platform database, the problem of low query efficiency for massive amounts of data has been solved, and a more efficient data filtering and caching mechanism has been achieved, thereby improving query speed.

CN116932544BActive Publication Date: 2026-02-17CHINA TELECOM CORP LTD TECHNOLOGY INNOVATION CENTER +1
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202310832236.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-07-06
Publication Date
2026-02-17
Estimated Expiration
2043-07-06

AI Technical Summary

Technical Problem

When querying massive amounts of statistical data in the platform's database, the process is inefficient and makes it difficult to quickly filter out data that meets the user's needs.

Method used

A three-tier storage structure is adopted, including a local database, a cache database, and an Elasticsearch database. The query strategy is determined based on the data type, queries are performed in different databases, and the query results are cached in the cache database to improve efficiency.

Benefits of technology

By using fine-grained partitioning and distributed storage, the need for massive searches in a single database is reduced, significantly improving data query efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116932544B_ABST
    Figure CN116932544B_ABST
Patent Text Reader

Abstract

The application discloses a data query method and device, electronic equipment and storage medium, and relates to the technical field of databases. The method comprises the following steps: receiving a query request from a platform, determining the data type requested by the query request; determining a query strategy in a data storage library according to the data type; wherein the data storage library comprises a local database, a cache database and an Elasticsearch database; and querying in the data storage library according to the query strategy, and obtaining target statistical data. Through the three-layer storage structure, massive statistical data is finely divided, and is stored in a scattered and summarized manner. In the query process, massive search in a database is not required, and the data query efficiency is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of database technology, and in particular to a data query method, apparatus, electronic device and storage medium. Background Technology

[0002] Against the backdrop of the leapfrog development of industrial digitalization, the sharing and integration of technologies across industries has become an inevitable choice for enterprises to migrate to the cloud and achieve digital transformation. Enterprises will leverage platforms (such as atomic capability platforms) to achieve enterprise construction, operation, and management. Atomic capabilities, relying on resources and long-term accumulation in cloud networking, IT, security, applications, and the Internet of Things, are formed through internal capability encapsulation and external capability introduction. These capabilities can be independently provided by enterprises and possess characteristics such as integrability (standardization), reusability, pricing flexibility, scalability, licensability, and broad common needs. Their primary service models are integration and being integrated.

[0003] Because the platform generates various statistical data every day, over time, the database configured on the platform (e.g., Elasticsearch database) will store more and more statistical data. Therefore, when users make queries, they need to filter out the statistical data that meets their needs from the massive amount of statistical data, which undoubtedly increases the workload of the query and leads to low efficiency. Summary of the Invention

[0004] This application provides a data query method to improve the efficiency of data query.

[0005] Firstly, a data query method is provided, including:

[0006] The system receives a query request from the platform and determines the data type requested. Based on the data type, it determines a query strategy in the data repository, which includes a local database, a cached database, and an Elasticsearch database. The system then performs a query in the data repository according to the query strategy and obtains the target statistical data.

[0007] In one possible implementation, the local database includes a first statistical statistic, which is a statistical statistic without a time variable; the Elasticsearch database includes the first statistical statistic and a second statistical statistic, which is a statistical statistic with a time variable; and the cache database includes a portion of the statistical statistic from the second statistical statistic.

[0008] In one possible implementation, determining the query strategy in the data repository based on the data type includes:

[0009] When it is determined that the data type requested by the query request is without a time variable, the query strategy in the data repository is to query in the local database; or when it is determined that the data type requested by the query request is with a time variable, the query strategy in the data repository is to query in the cache database and / or in the Elasticsearch database, wherein the cache database has a higher priority than the Elasticsearch database.

[0010] In one possible implementation, after querying the data repository according to the query strategy and obtaining the target statistics, the process further includes:

[0011] If all the target statistical data are obtained by querying the Elasticsearch database, the target statistical data are copied and cached in the cache database; or if some of the statistical data in the target statistical data are obtained by querying the cache database, and the remaining statistical data in the target statistical data are obtained by querying the Elasticsearch database, the remaining statistical data are copied and cached in the cache database.

[0012] In one possible implementation, the method further includes:

[0013] The system receives a deletion request from the platform, the deletion request being used to request the deletion of invalid statistical data in the cache database; according to the set aggregation rules, it identifies the invalid statistical data in the cache database and deletes the invalid statistical data from the cache database.

[0014] Secondly, a data query device is provided, comprising:

[0015] A receiving module is used to receive query requests from the platform; a determining module is used to determine the data type requested by the query request; and based on the data type, to determine a query strategy in the data repository; wherein the data repository includes a local database, a cache database, and an Elasticsearch database; and a query module is used to perform a query in the data repository according to the query strategy and obtain the target statistical data.

[0016] In one possible implementation, the local database includes a first statistical statistic, which is a statistical statistic without a time variable; the Elasticsearch database includes the first statistical statistic and a second statistical statistic, which is a statistical statistic with a time variable; and the cache database includes a portion of the statistical statistic from the second statistical statistic.

[0017] In one possible implementation, the determining module is specifically used for:

[0018] When it is determined that the data type requested by the query request is without a time variable, the query strategy in the data repository is to query in the local database; or when it is determined that the data type requested by the query request is with a time variable, the query strategy in the data repository is to query in the cache database and / or in the Elasticsearch database, wherein the cache database has a higher priority than the Elasticsearch database.

[0019] In one possible implementation, the device further includes a replication module;

[0020] The replication module is used to replicate the target statistical data and cache it in the cache database when all the target statistical data are obtained from the Elasticsearch database; or to replicate the remaining statistical data and cache it in the cache database when some of the target statistical data are obtained from the cache database and the remaining statistical data are obtained from the Elasticsearch database.

[0021] In one possible implementation, the apparatus further includes a deletion module; the receiving module is further configured to receive a deletion request from the platform, the deletion request being for requesting the deletion of invalid statistical data in the cache database; the deletion module is configured to determine the invalid statistical data in the cache database according to a set aggregation rule, and delete the invalid statistical data from the cache database.

[0022] Thirdly, an electronic device is provided, comprising:

[0023] A memory for storing computer programs; a processor for executing the computer programs stored in the memory to implement the method steps described in any one of the first aspects.

[0024] Fourthly, a computer-readable storage medium is provided, wherein a computer program is stored therein, and when executed by a processor, the computer program implements the steps of the method described in any one of the first aspects.

[0025] In this embodiment, a query request from the platform is received, and the data type of the query request is determined. Based on the data type, a query strategy is determined in the data repository. The data repository includes a local database, a cache database, and an Elasticsearch database. The query is performed in the data repository according to the query strategy, and the target statistical data is obtained. Therefore, through the above three-layer storage structure, massive statistical data is divided into fine-grained parts and stored in a distributed and summarized manner. During the query process, there is no need to perform massive searches in a single database, thus improving data query efficiency.

[0026] For the various aspects of the second to fourth aspects mentioned above, and the technical effects that each aspect may achieve, please refer to the above description of the technical effects that can be achieved for the first aspect or the various possible solutions in the first aspect, which will not be repeated here. Attached Figure Description

[0027] Figure 1 This is a schematic diagram illustrating the application scenarios applicable to the embodiments of this application;

[0028] Figure 2 This is a schematic diagram of the structure of a storage database provided in an embodiment of this application;

[0029] Figure 3 A flowchart illustrating a data query method provided in this application embodiment;

[0030] Figure 4 This is a schematic diagram of the structure of a data query device provided in an embodiment of this application;

[0031] Figure 5 This is a schematic diagram of another data query device provided in an embodiment of this application;

[0032] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The specific operational methods in the method embodiments can also be applied to the device embodiments or system embodiments. It should be noted that in the description of this application, "multiple" is understood as "at least two". "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. A connected to B can represent: A and B directly connected, and A and B connected through C. Furthermore, in the description of this application, terms such as "first" and "second" are used only for distinguishing the purpose of description and should not be construed as indicating or implying relative importance or order.

[0034] To facilitate understanding by those skilled in the art, the technical terms involved in the embodiments of this application will first be explained.

[0035] (1) Statistical data without time variables refers to all statistical data since its inception or statistical data at a fixed time (e.g., statistical data within 24 hours).

[0036] (2) Statistical data with time variables refers to statistical data with a non-fixed time dimension, where the time can be customized in each query.

[0037] (3) Elasticsearch is a distributed, highly scalable, and real-time search and data analysis engine suitable for all types of data, including text, numbers, geospatial data, structured data and unstructured data.

[0038] The following is a brief introduction to the application scenarios to which the technical solutions of the embodiments of this application are applicable. It should be noted that the application scenarios described below are only for illustrating the embodiments of this application and are not intended to limit the scope. In specific implementation, the technical solutions provided by the embodiments of this application can be flexibly applied according to actual needs.

[0039] Figure 1 This is a schematic diagram illustrating an application scenario applicable to the embodiments of this application. The scenario mainly includes a terminal device 101 and a database server 102. The terminal device 101 and the database server 102 can interact via a communication network, which can employ wireless communication or wired communication methods.

[0040] For example, terminal device 101 can access the network and communicate with database server 102 via cellular mobile communication technology, which may include 5th generation mobile network (5G) technology.

[0041] For example, terminal device 101 can access the network and communicate with database server 102 via short-range wireless communication, which may include Wireless Fidelity (Wi-Fi) technology.

[0042] This application embodiment does not impose any limitation on the number of the above-mentioned devices, such as Figure 1 As shown, only terminal device 101 and database server 102 are described as examples. The following is a brief introduction to each of the above devices and their respective functions.

[0043] Terminal device 101 is a device that can provide voice and / or data connectivity to users, including: handheld terminal devices with wireless connectivity, vehicle-mounted terminal devices, etc.

[0044] For example, terminal device 101 includes, but is not limited to: mobile phones, tablets, laptops, handheld computers, mobile internet devices (MID), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminal devices in industrial control, wireless terminal devices in autonomous driving, wireless terminal devices in smart grids, wireless terminal devices in transportation safety, wireless terminal devices in smart cities, or wireless terminal devices in smart homes, etc.

[0045] Furthermore, a data query-related platform can be installed on the terminal device 101. This platform can be software (e.g., an app, a browser, etc.), or a webpage, a mini-program, etc. In this embodiment, the terminal device 101 can use the aforementioned data query-related platform and, after exchanging information with the database server 102 to perform statistical data query, provide the corresponding results to the user.

[0046] Furthermore, the database server 102 can be equipped with a data repository corresponding to the aforementioned platform to provide statistical data required for various query requests. This data repository can consist of multi-layered databases, each storing statistical data across different dimensions and time ranges, such as... Figure 2 As shown:

[0047] Figure 2 This is a schematic diagram of the structure of a data repository provided in an embodiment of this application. The data repository 200 includes a local database 200a, a cache database 200b, and an Elasticsearch database 200c.

[0048] Local database 200a can be used to store statistical data without time variables, and the statistical data in local database 200a can be updated according to a set period.

[0049] The cache database 200b can be used to cache partial statistics without time variables, broken down into time granularities. For example, this partial statistics could be data that a user consistently uses or that is used frequently.

[0050] Elasticsearch Database 200c can be used to store all statistical data without time variables and statistical data with time variables, facilitating data retrieval and fallback.

[0051] In some embodiments, the databases in the storage database 200 may also be configured with a priority order, for example, local database 200a > cache database 200b > Elasticsearch database 200c; or, for example, local database 200a = cache database 200b > Elasticsearch database 200c, so that the search starts from a small range during the query process, thereby speeding up the data query efficiency.

[0052] In this embodiment, the massive statistical data is divided into fine-grained categories and stored in a distributed and summarized manner through the three-layer storage structure provided above. During the query process, the platform can call any one or more databases in the data repository 200 and provide the user with the statistical data they want to query. Compared with performing a massive search in a single database, this speeds up the data query efficiency.

[0053] To further illustrate the technical solutions provided in the embodiments of this application, a detailed description is provided below in conjunction with the accompanying drawings and specific implementation methods. Although the embodiments of this application provide method operation steps as shown in the following embodiments or drawings, the method may include more or fewer operation steps based on conventional or non-inventive methods. In steps where there is no logically necessary causal relationship, the execution order of these steps is not limited to the execution order provided in the embodiments of this application. In actual processing or when the device executes the method, it may be executed in the order shown in the embodiments or drawings, or in combination.

[0054] Figure 3 This is a flowchart illustrating a data query method provided in an embodiment of this application. This process can be executed by a data query device, which can be implemented in software, hardware, or a combination of both, to improve data query efficiency. Figure 3 As shown, the process includes the following steps:

[0055] 301: Receives a query request from the platform and determines the data type of the query request.

[0056] In one possible implementation, the query request may carry information such as time range, data identifier, data name, and data size to facilitate determining the data type used in the query request.

[0057] 302: Based on the data types described above, determine the query strategy in the data repository.

[0058] The data repository (such as) Figure 1 The data repository 200 shown includes local databases (such as...) Figure 2 The local database 200a shown), cache database (such as...) Figure 2 The local database 200b shown), Elasticsearch database (such as...) Figure 2 The Elasticsearch database shown is 200c.

[0059] In one possible implementation, the three databases described above can store different types of statistical data. For example, the local database includes first statistical data, which is statistical data without a time variable; the Elasticsearch database includes first statistical data and second statistical data, which is statistical data with a time variable; and the cache database includes a portion of the statistical data in the second statistical data, thereby classifying and distributing the massive amount of statistical data to facilitate data retrieval by users.

[0060] In one possible implementation, determining the query strategy in the data repository can include the following:

[0061] Scenario 1: When it is determined that the data type of the query request is without time variables, the query strategy in the data repository is to query the local database. Therefore, the search can be performed specifically on statistical data without time variables, which speeds up the data query efficiency.

[0062] Scenario 2: When it is determined that the data type of the query request is time-varying, the query strategy in the data repository is determined to be to perform the query in at least one of the cache database and the Elasticsearch database according to the priority rule, as follows:

[0063] When it is determined that the data type of the query request is time-varying and the time range is within the set range, the query strategy in the data repository is to query the cache database.

[0064] When it is determined that the data type of the query request is time-varying and the time range is not within the set range, the query strategy in the data repository is to query the Elasticsearch database.

[0065] When it is determined that the data type of the query request is time-varying, and part of the time range is within the set range while another part is outside the set range, the query strategy in the data repository is to first query the cache database and then query the Elasticsearch database.

[0066] 303: The query is performed in the data repository according to the determined query strategy, and the target statistical data is obtained.

[0067] In this step: when the query strategy is determined to be to query in the local database, the local database can be directly called, and the target statistical data can be filtered from the first statistical data based on the time range and data identifier. The data is then fed back to the platform and displayed to the user.

[0068] Once the query strategy is determined to be to query the Elasticsearch database, the Elasticsearch database can be directly invoked. The target statistical data can be filtered from the first statistical data based on the time range and data identifier, and then fed back to the platform and displayed to the user.

[0069] When the query strategy is determined to be to query the cache database, the cache database can be directly invoked, and the target statistical data can be filtered from the first statistical data based on the time range and data identifier. The results are then fed back to the platform and displayed to the user.

[0070] When the query strategy is determined to be to query in both the cache database and the Elasticsearch database, the cache database can be called first to filter out a portion of the target statistical data based on the time range and data identifier. Then, the Elasticsearch database can be called to filter out another portion of the target statistical data based on the time range and data identifier. The results are then fed back to the platform and displayed to the user.

[0071] In one possible implementation, after retrieving the target statistical data, further analysis can be performed to determine whether to copy the target statistical data and cache it in a cache database. This allows for direct retrieval from the cache database during subsequent data queries, further accelerating data retrieval efficiency. Specifically, this can include the following scenarios:

[0072] Scenario 1: When all target statistics are obtained from the Elasticsearch database, copy the target statistics and cache them in the cache database.

[0073] Scenario 2: When some of the statistical data in the target statistics are obtained from the cache database, and the remaining statistical data in the target statistics are obtained from the Elasticsearch database, copy the remaining statistical data and cache it in the cache database.

[0074] In other embodiments, statistical data with time variables are divided into time granularities, and the amount of statistical data stored in the cache database may be excessive. Since many statistical data (e.g., security attack data) are not uniformly distributed over time, the statistical data in the cache database can be aggregated to identify invalid statistical data and delete it, thereby reducing the number of data entries in the cache database. The specific method is as follows:

[0075] The system receives deletion requests from the platform, requesting the removal of invalid statistical data from the cache database. Based on predefined aggregation rules, it identifies and removes these invalid statistical data from the cache database. For example, as shown in Table 1 below, assuming the statistical data has a daily time granularity, storing it daily might result in some invalid statistical data. Therefore, based on aggregation rules, the statistical data in Table 1 with times of 2023-03-27 and 2023-03-28 can be identified as invalid and removed from the cache database, resulting in the aggregated statistical data shown in Table 2. This allows for real-time cleanup of the cache database's memory, reducing resource consumption.

[0076] Table 1: Example table of statistical data before aggregation in the cache database

[0077] time Parameter 1 Parameter 2 quantity 2023-3-26 XX YY 27 2023-3-27 XX YY 0 2023-3-28 XX YY 0 2023-3-29 XX YY 16

[0078] Table 2: Example table of aggregated statistical data in the cache database

[0079] time Parameter 1 Parameter 2 quantity 2023-3-26 XX YY 27 2023-3-29 XX YY 16

[0080] In this embodiment, a query request from the platform is received, and the data type of the query request is determined. Based on the data type, a query strategy is determined in the data repository. The data repository includes a local database, a cache database, and an Elasticsearch database. The query is performed in the data repository according to the query strategy, and the target statistical data is obtained. Therefore, through the above three-layer storage structure, massive statistical data is divided into fine-grained parts and stored in a distributed and summarized manner. During the query process, there is no need to perform massive searches in a single database, thus improving data query efficiency.

[0081] Based on the same technical concept, this application also provides a data query device, which can implement the data query method flow described above in this application.

[0082] Figure 4 This is a schematic diagram of a data query device provided in an embodiment of this application. The device includes: a receiving module 401, a determining module 402, and a query module 403.

[0083] The receiving module 401 is used to receive query requests from the platform.

[0084] The determination module 402 is used to determine the data type of the query request; and to determine the query strategy in the data repository based on the data type; wherein the data repository includes a local database, a cache database, and an Elasticsearch database.

[0085] The query module 403 is used to perform a query in the data repository according to the query strategy and obtain the target statistical data.

[0086] In one possible implementation, the determining module 402 is specifically used for:

[0087] When it is determined that the data type requested by the query request is without a time variable, the query strategy in the data repository is to query in the local database; or when it is determined that the data type requested by the query request is with a time variable, the query strategy in the data repository is to query in the cache database and / or in the Elasticsearch database, wherein the cache database has a higher priority than the Elasticsearch database.

[0088] In other embodiments, in addition to the above... Figure 4 In addition to the modules shown, it may further include a copy module and a delete module, such as... Figure 5As shown, an exemplary schematic diagram of another data query device provided in an embodiment of this application is illustrated. The device includes: a receiving module 401, a determining module 402, a query module 403, a copying module 501, and a deleting module 502.

[0089] The replication module 501 is used to replicate the target statistical data and cache it in the cache database when all the target statistical data are obtained by querying the Elasticsearch database; or to replicate the remaining statistical data and cache it in the cache database when some of the target statistical data are obtained by querying the cache database and the remaining statistical data are obtained by querying the Elasticsearch database.

[0090] The receiving module 401 is further configured to receive a deletion request from the platform, the deletion request being used to request the deletion of invalid statistical data in the cache database; the deletion module 502 is configured to determine the invalid statistical data in the cache database according to the set aggregation rules, and delete the invalid statistical data from the cache database.

[0091] It should be noted that the apparatus provided in this application embodiment can implement all the method steps in the above method embodiment and achieve the same technical effect. Therefore, the parts and beneficial effects that are the same as those in the method embodiment will not be described in detail here.

[0092] Based on the same technical concept, this application also provides an electronic device that can realize the function of the aforementioned data query device.

[0093] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0094] At least one processor 601 and a memory 602 connected to at least one processor 601. In this embodiment, the specific connection medium between the processor 601 and the memory 602 is not limited. Figure 6 The example shown is the connection between processor 601 and memory 602 via bus 600. Bus 600 is... Figure 6 The connections between other components are indicated by thick lines and are for illustrative purposes only, not as limiting information. The 600 bus can be divided into address bus, data bus, control bus, etc., for ease of representation. Figure 6 The term is represented by a single thick line, but this does not imply that there is only one bus or one type of bus. Alternatively, the processor 601 can also be called a controller; there is no restriction on the name.

[0095] In this embodiment, memory 602 stores instructions executable by at least one processor 601. By executing the instructions stored in memory 602, at least one processor 601 can perform a data query method as described above. Processor 601 can implement... Figure 4 or Figure 5 The functions of each module in the device shown.

[0096] The processor 601 is the control center of the device. It can connect to various parts of the control device through various interfaces and lines. By running or executing instructions stored in memory 602 and calling data stored in memory 602, the processor can perform various functions and process data, thereby monitoring the device as a whole.

[0097] In one possible design, processor 601 may include one or more processing units. Processor 601 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into processor 601. In some embodiments, processor 601 and memory 602 may be implemented on the same chip; in some embodiments, they may also be implemented on separate chips.

[0098] Processor 601 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor, application-specific integrated circuit, field-programmable gate array or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of a data query method disclosed in the embodiments of this application can be directly manifested as execution by a hardware processor, or execution by a combination of hardware and software modules within the processor.

[0099] Memory 602, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory 602 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage, magnetic disk, optical disk, etc. Memory 602 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto. In the embodiments of this application, memory 602 may also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.

[0100] By designing and programming the processor 601, the code corresponding to a data query method described in the foregoing embodiments can be embedded into the chip, enabling the chip to execute it during operation. Figure 3 The illustrated embodiment presents a data query method. How to design and program the processor 601 is a technique well-known to those skilled in the art and will not be described further here.

[0101] It should be noted that the electronic device provided in this application embodiment can implement all the method steps implemented in the above method embodiment and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiment and the beneficial effects will not be described in detail.

[0102] This application also provides a computer-readable storage medium storing computer-executable instructions for causing a computer to execute a data query method described in the above embodiments.

[0103] This application also provides a computer program product, which, when invoked by a computer, causes the computer to execute a data query method described in the above embodiments.

[0104] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0105] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0106] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0107] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

Claims

1. A data query method, characterized by, The method comprises: receiving a query request from a platform, and determining a data type requested by the query request; determining a query strategy in a data repository according to the data type, wherein the data repository comprises a local database, a cache database and an Elasticsearch database, the local database comprises first statistical data, the first statistical data being time-variable-free statistical data, the Elasticsearch database comprises the first statistical data and second statistical data, the second statistical data being time-variable statistical data, and the cache database comprises part of the second statistical data; querying in the data repository according to the query strategy, and obtaining target statistical data through the querying; wherein the determining the query strategy in the data repository according to the data type comprises: when it is determined that the data type requested by the query request is time-variable-free, determining that the query strategy in the data repository is to query in the local database; or when it is determined that the data type requested by the query request is time-variable, determining that the query strategy in the data repository is to query in the cache database and / or the Elasticsearch database, the priority of the cache database being higher than that of the Elasticsearch database.

2. The method of claim 1, wherein, After the querying in the data repository according to the query strategy and the obtaining of the target statistical data, the method further comprises: when the target statistical data are all obtained in the Elasticsearch database, copying and caching the target statistical data in the cache database; or when part of the target statistical data are obtained in the cache database and the remaining target statistical data are obtained in the Elasticsearch database, copying and caching the remaining target statistical data in the cache database.

3. The method of claim 1, wherein, The method further comprises: receiving a deletion request from the platform, the deletion request being used to request to delete invalid statistical data in the cache database; determining the invalid statistical data in the cache database according to a set aggregation rule, and deleting the invalid statistical data from the cache database.

4. A data query apparatus, characterized by comprising: The method comprises: a receiving module configured to receive a query request from a platform; a determining module configured to determine a data type requested by the query request; and determine a query strategy in a data repository according to the data type, wherein the data repository comprises a local database, a cache database and an Elasticsearch database, the local database comprises first statistical data, the first statistical data being time-variable-free statistical data, the Elasticsearch database comprises the first statistical data and second statistical data, the second statistical data being time-variable statistical data, and the cache database comprises part of the second statistical data. The query module is configured to query the data storage according to the query strategy and obtain target statistical data. The determination module is configured to: when determining that the data type requested by the query request is time-variable, determine that the query strategy in the data storage is to query in the local database; or when determining that the data type requested by the query request is time-variable, determine that the query strategy in the data storage is to query in the cache database and / or the Elasticsearch database, and the priority of the cache database is higher than that of the Elasticsearch database.

5. The apparatus of claim 4, wherein, The apparatus further comprises a replication module. The replication module is configured to: when all the target statistical data are obtained by querying the Elasticsearch database, replicate the target statistical data and cache the target statistical data in the cache database; or when part of the target statistical data is obtained by querying the cache database and the remaining target statistical data is obtained by querying the Elasticsearch database, replicate the remaining target statistical data and cache the remaining target statistical data in the cache database.

6. The apparatus of claim 4, wherein, The apparatus further comprises a deletion module. The receiving module is further configured to receive a deletion request from the platform, the deletion request being used to request to delete invalid statistical data in the cache database. The deletion module is configured to determine invalid statistical data in the cache database according to a set aggregation rule, and delete the invalid statistical data from the cache database.

7. An electronic device, comprising: The apparatus comprises: a memory configured to store a computer program; a processor configured to execute the computer program stored in the memory, and implement the method steps in any one of claims 1-3.

8. A computer-readable storage medium, characterized in that, The computer program stored in the computer readable storage medium is executed by the processor, and the method steps in any one of claims 1-3 are implemented.

Citation Information

Patent Citations

  • Four-layer structure data acquisition method and device based on distributed system

    CN111694865A

  • Meteorological data service platform based on micro-service architecture

    CN113904917A