A cloud computing-based data retrieval method, device and system
By adaptively generating search request fields, and utilizing scheduling services and multiple search engines for cloud database data retrieval, the error problem caused by inconsistent search conditions is solved, thereby improving accuracy and efficiency and reducing development and maintenance costs.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA UNIONPAY
- Filing Date
- 2022-08-23
- Publication Date
- 2026-06-05
Smart Images
Figure CN115438032B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data processing technology, and in particular to a data retrieval method, apparatus and system based on cloud computing. Background Technology
[0002] With the surge in online information resources and the increasing variety of resource types, the amount of data requiring storage is growing. Following the introduction of cloud computing, data previously stored offline has gradually shifted to online storage, specifically to cloud databases.
[0003] To prevent data silos when users search for data, it is necessary to retrieve data from various cloud databases and then integrate the data. Currently, the method for retrieving data from various cloud databases is generally to send the search request directly to each search engine; each search engine interfaces with different cloud databases, and each search engine loads a unified search service. When users perform data searches, each search engine calls the unified search service to retrieve data results from each cloud database, and then the data results are integrated.
[0004] However, because different enterprises manage data differently, the data formats of various cloud databases also differ, leading to variations in the search criteria used when retrieving data. Standardizing the data retrieval methods across different cloud data sources would result in consistent search criteria, potentially causing errors. Furthermore, implementing a unified search service presents challenges in business logic development, including high development difficulty and maintenance costs. Summary of the Invention
[0005] This invention provides a cloud computing-based data retrieval method, apparatus, and system for adaptively retrieving data results from various cloud databases, improving the accuracy of data retrieval, avoiding the need for redevelopment of business logic, and reducing the difficulty and maintenance cost of business logic development.
[0006] In a first aspect, embodiments of the present invention provide a data retrieval method based on cloud computing, comprising:
[0007] Obtain a search request; the search request includes a first request field;
[0008] Based on the first request field, a second request field that meets the search criteria is generated; the first request field and the second request field are different; the search criteria are generated based on fields in the cloud database, representing fields in the cloud database that are allowed to be indexed.
[0009] Data is retrieved from the cloud database based on the second request field to obtain the first search result.
[0010] In the above technical solution, the cloud-based data retrieval method is applied to a data retrieval system, which includes a scheduling service and multiple search engines. The search criteria for each search engine are different. For any given search engine, the search criteria are generated based on fields in the cloud database that the search engine interfaces with, representing the fields in the cloud database that are allowed to be indexed. The scheduling service is used to obtain search requests sent by users and, after obtaining the search requests, sends them to each search engine, which then retrieves data from its corresponding cloud database.
[0011] For any given search engine, there are pre-defined search criteria. The search engine generates a second request field based on the first request field, according to the corresponding search criteria. If the second request field satisfies the search criteria, the accuracy of the retrieved data is guaranteed when data is retrieved from the cloud database based on the second request field. Furthermore, it eliminates the need to set up search services for each search engine, avoiding the need for redevelopment of business logic and reducing the difficulty and maintenance costs of business logic development.
[0012] Optionally, after obtaining the first search result, the following may also be included:
[0013] Based on cloud-native traffic transfer records, retrieve homogeneous data from the first search result;
[0014] After deduplicating the homogeneous data in the first search result, the deduplicated first search result is cached in the first-level cache space to obtain the first-level cache data; the first-level cache space includes a first condition sequence table; the first condition sequence table records the mapping relationship between the first request field and the first-level index; the first-level index is used to index the first-level cache data.
[0015] The above technical solution improves the accuracy of data retrieval by deduplicating homogeneous data and avoiding duplicate data in the search results.
[0016] Optionally, the first deduplicated search result is cached in the first-level cache space to obtain the first-level cache data, including:
[0017] Calculate the capacity of the first-level cache data;
[0018] Sort and group the fields in the first request field;
[0019] Sort the first deduplicated search results according to the sorting and grouping of each field;
[0020] Based on the capacity of the first-level cache data, the sorted first search result is cached in the first-level cache space.
[0021] Optionally, calculating the capacity of the first-level cache data includes:
[0022] Query the capacity of the first-level cache space and the capacity of the first search result;
[0023] The capacity of the cached first-level cache data is calculated based on the capacity of the first-level cache space and the capacity of the first search result; the capacity of the first-level cache data is directly proportional to the capacity of the first-level cache space; the capacity of the first-level cache data is inversely proportional to the capacity of the first search result.
[0024] In the above technical solution, the capacity of the first-level cache data is adjusted in real time and adaptively to avoid wasting computer resources and improve data retrieval efficiency.
[0025] Optionally, after obtaining the first-level cache data, the following may also be included:
[0026] Based on the first request field, data is retrieved from the first-level cache data to obtain the second retrieval result;
[0027] The second search result is cached in the second-level cache space to obtain the second-level cache data; the second-level cache space includes a second condition sequence table; the second condition sequence table records the mapping relationship between the first request field and the second-level index; the second-level index is used to index the second-level cache data;
[0028] Feedback is provided on the secondary cache data.
[0029] Optionally, the method further includes:
[0030] Receive an eviction instruction for L1 cache data; the eviction instruction is sent after the L1 cache data has been modified;
[0031] Delete the second cache data corresponding to the elimination instruction from the second-level cache space;
[0032] Remove the first cached data that was not hit within the preset period from the first-level cache space.
[0033] In the above technical solution, the secondary cache data is adjusted in real time to ensure consistency between the primary and secondary cache data. By deleting cached data that has not been cached within a preset period from the primary cache space, waste of computer resources is avoided.
[0034] Secondly, embodiments of the present invention provide a cloud computing-based data retrieval device, comprising:
[0035] The acquisition module is used to acquire a search request; the search request includes a first request field;
[0036] The processing module is used to generate a second request field that meets the search conditions based on the first request field; the first request field and the second request field are different; the search conditions are generated based on fields in the cloud database, representing fields in the cloud database that are allowed to be indexed.
[0037] Data is retrieved from the cloud database based on the second request field to obtain the first search result.
[0038] Thirdly, embodiments of the present invention provide a cloud computing-based data retrieval system, a scheduling service, and multiple retrieval engines; the retrieval conditions of the multiple retrieval engines are different; for any retrieval engine, the retrieval conditions of the retrieval engine are generated based on fields of data in a cloud database that is connected to the retrieval engine, representing fields in the cloud database that allow indexing of data;
[0039] The scheduling service is used to obtain a search request and send the search request to the search engine; the search request includes a first request field;
[0040] For any search engine, the search engine is used to generate a second request field that satisfies the search conditions of the search engine based on the first request field; the first request field and the second request field are different.
[0041] The search engine retrieves data from the cloud database based on the second request field to obtain the first search result.
[0042] Fourthly, embodiments of the present invention also provide a computer device, comprising:
[0043] Memory, used to store program instructions;
[0044] The processor is used to call the program instructions stored in the memory and execute the above-mentioned cloud computing-based data retrieval method according to the obtained program.
[0045] Fifthly, embodiments of the present invention also provide a computer-readable storage medium storing computer-executable instructions for causing a computer to execute the above-described cloud computing-based data retrieval method. Attached Figure Description
[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0047] Figure 1 This invention provides an architectural diagram of a cloud computing-based data retrieval system according to an embodiment of the present invention.
[0048] Figure 2 A flowchart illustrating a cloud computing-based data retrieval method provided in an embodiment of the present invention;
[0049] Figure 3 A schematic diagram of the structure of a search engine provided in an embodiment of the present invention;
[0050] Figure 4 This is a schematic diagram of a cloud computing-based data retrieval device provided in an embodiment of the present invention. Detailed Implementation
[0051] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.
[0052] Figure 1 An exemplary schematic diagram of the architecture of a cloud computing-based data retrieval system applicable to an embodiment of the present invention is shown. The system architecture includes a scheduling service 110 and a retrieval engine 120.
[0053] The scheduling service 110 is used to obtain a search request and send it to the search engine 120. The search request includes a first request field. For example, the scheduling service obtains the user's search request through an API (Application Programming Interface). The request field represents the field to be searched, such as height, weight, order number, etc. The request field can include multiple fields, such as fields a, b, and c.
[0054] Search engine 120 includes multiple search engines, such as search engine j1, search engine j2, ..., search engine jn, where n represents the number of search engines. The search engines interface with the cloud database to retrieve data from it. Correspondingly, the cloud database and the search engine are paired, such as cloud database s1 interfaced with search engine j1; however, cloud database s1 can include multiple databases, which are not limited here.
[0055] For any given search engine, search criteria are set. Therefore, after receiving the first requested field, the search engine first determines whether the first requested field meets the search criteria. If not, it generates a second requested field that meets the search criteria based on the first requested field. For example, the first requested field may contain three fields: a, b, and c. The cloud database that the search engine connects to does not have field c, therefore field c does not meet the search engine's search criteria. Then, based on the first requested field, a second requested field that meets the search criteria is generated; the second requested field only includes fields a and b.
[0056] After generating the second request field, the search engine retrieves data from the cloud database based on the second request field to obtain the first search result.
[0057] The search engine is used to integrate the initial search results from various search engines to obtain cached data. This cached data is then used by users for secondary searches.
[0058] It should be noted that the above Figure 1 The structure shown is merely an example, and the embodiments of the present invention are not limited thereto.
[0059] Based on the above description Figure 2 An exemplary schematic diagram of a cloud-based data retrieval method provided by an embodiment of the present invention is shown, which can be executed by a cloud-based data retrieval device.
[0060] like Figure 2 As shown, the process specifically includes:
[0061] Step 210: Obtain the search request.
[0062] The search request includes a first request field. The search request is sent by the user, received by the scheduling service, and then forwarded to the respective search engines. For example, the user sends a search request to the scheduling service via a UI (User Interface) or CLI (Command-line Interface). The first request field of the search request can include multiple fields; for example, it could include four fields: a, b, c, and d, representing the province, city, county, and town, respectively. The search request indicates a search for the population of province a, city b, county c, and town d.
[0063] Step 220: Generate a second request field that meets the search conditions based on the first request field.
[0064] The first and second request fields are different; the search criteria are generated based on fields in the cloud database, representing the fields in the cloud database that are allowed to be indexed. For example, if cloud database s1 stores data in "key-value" format, and the primary key "key" includes three fields: a, b, and c, then cloud database s1 allows indexing of fields a, b, and c. In other words, the search criteria for search engine j1 are: the requested field falls within the range of fields a, b, and c.
[0065] In some feasible approaches, the search criteria of each search engine differ, and the first requested field does not meet the search criteria of each search engine. For example, search engine j1 connects to cloud database s1; cloud database s1 contains data records of the population of county c in city b of province a, excluding the population of towns and areas under county c.
[0066] In other words, the search criteria of search engine j1 can only query three fields: a, b, and c. Therefore, based on the first request field, search engine j1 generates a second request field that satisfies the search criteria of the search engine, that is, the second request field includes the three fields a, b, and c.
[0067] In some feasible approaches, if the first request field satisfies the search criteria of each search engine, then the first request field is directly used as the second request field.
[0068] Step 230: Retrieve data from the cloud database based on the second request field to obtain the first search result.
[0069] Figure 3 This is a schematic diagram illustrating the structure of a search engine exemplarily shown in this application, such as... Figure 3 As shown, the search engine includes a search dispatch module, a search management module, a cache management module, and a cache search module. The search dispatch module generates a second request field that satisfies the search engine's search conditions based on a first request field and search criteria. The search management module retrieves data from the cloud database based on the second request field and adds the first search result to the first-level cache space. The cache search module retrieves data from the first-level cache data based on the first request field and caches the second search result to the second-level cache space. The cache management module provides feedback on the second-level cache data and adjusts the first and second cache data in real time.
[0070] Specifically, the search engine caches the first search result in the first-level cache space to obtain the first-level cache data; the first-level cache space includes the first condition sequence table; the first condition sequence table records the mapping relationship between the first requested field and the first-level index; the first-level index is used to index the first-level cache data.
[0071] The first conditional sequence list is shown in Table 1 below:
[0072] Table 1
[0073] First request field First Index cond_1 index_1 cond_2 index_2 …… …… cond_m index_m
[0074] The data for the first-level index and its cache is shown in Table 2 below:
[0075] Table 2
[0076] First Index Level 1 cache data index_1 data_1 index_2 data_2 …… …… index_m data_m
[0077] The first request field is an ordered sequence of fields organized based on the feature fields. The core attributes of the conditional features are the unique ID, type, and supported operators of the feature field. The assembly process sorts the conditional fields by their unique ID values and then concatenates them. The mapping relationship between each field and the first index is stored in the first-level cache space.
[0078] In some feasible implementations, the first request field includes multiple fields; the search engine is also used to sort and group the multiple fields; and the first search result is sorted according to the sorting and grouping of the multiple fields.
[0079] In other words, the search engine is also used to clean the conditions of the first request field. Condition cleaning refers to converting complex natural language search fields of various forms into the standard search fields of the search engine; for example, the natural language search field "thin body type" is converted into the standard search fields of the search engine such as "weight" and "BMI index".
[0080] The cleaning process includes field encoding, field sorting, and field grouping. Among them, conditional encoding refers to matching the user-input request field with the unique ID of the feature field in the standard field. Each conditional field will match a unique feature field.
[0081] Field sorting refers to sorting the features according to their field definitions, in ascending order of the numerical value of the feature field ID.
[0082] Field grouping refers to incorporating standard retrieval fields into different memory groups according to business domain planning, in order to group the first-level cache data.
[0083] In some embodiments, the search engine is further configured to query the capacity of the first-level cache space and the capacity of the first search result; calculate the capacity of the first-level cache data based on the capacity of the first-level cache space and the capacity of the first search result; the capacity of the first-level cache data is directly proportional to the capacity of the first-level cache space; and the capacity of the first-level cache data is inversely proportional to the capacity of the first search result.
[0084] The above technical solution adaptively controls the cache expansion rate exponentially, thereby managing the cache space. In some embodiments, temporarily unused L1 cache data can be placed in a high-speed disk array as temporary swap space; the specific method is not limited here.
[0085] For example, the search criteria for limiting the result set can be combined with the cache capacity to amplify the cache by 5-10 times before the search command is dispatched to the search engine. The specific amplification ratio is the proportion of the cache capacity to the total capacity, and it is linearly amplified, with a maximum of 10 times or other multiples, which is not specifically limited here.
[0086] After retrieving the first-level cache data, the search engine retrieves data from the first-level cache based on the first requested field, obtaining the second search result. This second search result is then cached in the second-level cache space, resulting in the second-level cache data. The second-level cache space includes a second condition sequence table, which records the mapping relationship between the first requested field and the second-level index. The second-level index is used to index the second-level cache data. Finally, the second-level cache data is displayed for the user to view.
[0087] The second conditional sequence list is shown in Table 3 below:
[0088] Table 3
[0089]
[0090]
[0091] The secondary index cache data is shown in Table 2 below:
[0092] Table 4
[0093] First Index Level 1 cache data rsindex_1 r_data_1 rsindex_2 r_data_2 …… …… rsindex_m r_data_m
[0094] The primary key and first-level cache data are standard field sequences. The difference between them and the first-level cache is that their corresponding values are the index values of the first search result. The index is used to retrieve the actual data from the first search result.
[0095] In this embodiment of the invention, the second-level cache data supports two types of retrieval: regular hash retrieval and retrieval based on fuzzy matching, sorting, and grouping. For regular hash retrieval, the data is directly retrieved from the hash table through two calculations. For retrieval involving fuzzy matching, sorting, and grouping, the first-level cache data is processed according to the corresponding requirements by extending special retrieval logic within the retrieval engine.
[0096] In some embodiments, the search engine is also configured to receive an eviction instruction for first-level cache data; wherein the eviction instruction is sent after the first-level cache data has been modified.
[0097] The search engine will remove the second cache data corresponding to the eviction instruction from the second-level cache space.
[0098] In other words, for secondary cache data, this embodiment of the invention provides a two-way feedback mechanism. Two-way feedback means that the secondary cache data receives requests from the primary cache data regarding cache data eviction, and simultaneously feeds back its own hit rate to the primary cache data. This ensures the consistency between the primary and secondary cache data. The hit rate refers to the hit rate when a user views the secondary cache data.
[0099] Because there is a logical delay in the hit rate response between Level 1 cache and Level 2 cache, the availability of Level 1 cache data is proactively probed when Level 2 cache data is not found, thus improving the accuracy of data retrieval. For example, if a data parameter in Level 1 cache changes from "1" to "2", and since Level 2 cache data is retrieved from Level 1 cache, assuming this data is indeed Level 2 cache data, the two caches are no longer consistent. Therefore, to ensure data consistency, the Level 2 cache data will be forcibly invalidated.
[0100] In addition, the granularity of invalidation handling for secondary cache data can be set by the device, which can distinguish between table-level, record-level, and field-level updates. When data that meets the conditions is updated, the corresponding secondary cache data is evicted.
[0101] In some embodiments, first-level cache data that has not been cached within a preset period is deleted from the first-level cache space. This frees up the first-level cache space, thereby reducing the consumption of computing resources.
[0102] For example, if a user does not view a certain part of the first cached data within 3 seconds (a preset period), then the first cached data of that part will be deleted from the first-level cache space.
[0103] In some feasible approaches, the search engine also retrieves homogeneous data from the first search results of multiple search engines based on cloud-native traffic transfer records; then, the homogeneous data from the first search results of multiple search engines is deduplicated and cached in the first-level cache space.
[0104] Homogeneous data refers to data that is completely identical. The integrated architecture of the cloud environment introduces the homogeneous data problem; the most typical example is that disaster recovery system data and primary data are exactly the same. Furthermore, different downstream application nodes (not the source nodes) may store essentially the same data content. Specifically, all traffic from disaster recovery nodes is transferred to the primary node. All traffic from downstream application nodes is transferred to the upstream application node (source node).
[0105] Therefore, homogeneous data from upstream application nodes and downstream nodes can be queried through traffic transfer records, and the homogeneous data from downstream nodes can be deleted to achieve the effect of deduplication.
[0106] Based on the same technological concept Figure 4 An exemplary schematic diagram of a cloud-based data retrieval device provided in an embodiment of the present invention is shown. This device can execute the process of a cloud-based data retrieval method.
[0107] like Figure 4 As shown, the device specifically includes:
[0108] The acquisition module 410 is used to acquire a search request; the search request includes a first request field;
[0109] Processing module 420 is used to generate a second request field that meets the search conditions based on the first request field; the first request field and the second request field are different; the search conditions are generated based on fields in the cloud database, representing fields in the cloud database that are allowed to be indexed.
[0110] Data is retrieved from the cloud database based on the second request field to obtain the first search result.
[0111] Optionally, the processing module 420 is further configured to:
[0112] After obtaining the first search result, based on the cloud-native traffic transfer record, query out the homogeneous data in the first search result;
[0113] After deduplicating the homogeneous data in the first search result, the deduplicated first search result is cached in the first-level cache space to obtain the first-level cache data; the first-level cache space includes a first condition sequence table; the first condition sequence table records the mapping relationship between the first request field and the first-level index; the first-level index is used to index the first-level cache data.
[0114] Optionally, the first request field includes multiple fields;
[0115] The processing module 420 is specifically used for:
[0116] Calculate the capacity of the first-level cache data;
[0117] Sort and group the fields in the first request field;
[0118] Sort the first deduplicated search results according to the sorting and grouping of each field;
[0119] Based on the capacity of the first-level cache data, the sorted first search result is cached in the first-level cache space.
[0120] Optionally, the processing module 420 is specifically used for:
[0121] Query the capacity of the first-level cache space and the capacity of the first search result;
[0122] The capacity of the cached first-level cache data is calculated based on the capacity of the first-level cache space and the capacity of the first search result; the capacity of the first-level cache data is directly proportional to the capacity of the first-level cache space; the capacity of the first-level cache data is inversely proportional to the capacity of the first search result.
[0123] Optionally, the processing module 420 is further configured to:
[0124] After obtaining the first-level cache data, data is retrieved from the first-level cache data based on the first request field to obtain the second retrieval result;
[0125] The second search result is cached in the second-level cache space to obtain the second-level cache data; the second-level cache space includes a second condition sequence table; the second condition sequence table records the mapping relationship between the first request field and the second-level index; the second-level index is used to index the second-level cache data;
[0126] Feedback is provided on the secondary cache data.
[0127] Optionally, the processing module 420 is further configured to:
[0128] Receive an eviction instruction for the L1 cache data; the eviction instruction is sent after the L1 cache data has been modified;
[0129] Delete the second cache data corresponding to the elimination instruction from the second-level cache space;
[0130] Remove the first cached data that was not hit within the preset period from the first-level cache space.
[0131] Based on the same technical concept, embodiments of the present invention also provide a computer device, including:
[0132] Memory, used to store program instructions;
[0133] The processor is used to call the program instructions stored in the memory and execute the above-mentioned cloud computing-based data retrieval method according to the obtained program.
[0134] Based on the same technical concept, embodiments of the present invention also provide a computer-readable storage medium storing computer-executable instructions for causing a computer to execute the above-described cloud computing-based data retrieval method.
[0135] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0136] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0137] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0138] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0139] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.
Claims
1. A data retrieval method based on cloud computing, characterized in that, Applied to a data retrieval system, the data retrieval system includes a scheduling service and multiple retrieval engines, each retrieval engine corresponding to a cloud database, the method is applied to each retrieval engine, and the method includes: The system receives a search request sent by the scheduling service; the search request includes a first request field; the scheduling service obtains the search request sent by the user and sends the search request to the plurality of search engines. The first request field is cleaned by conditions, which means converting complex natural language search fields of various forms into standard search fields of the search engine. Determine whether the first requested field satisfies the search conditions corresponding to the search engine; the search conditions are generated based on fields in the cloud database corresponding to the search engine, representing fields in the cloud database that are allowed to be indexed; the search conditions of the multiple search engines are different; If not satisfied, a second request field that satisfies the search conditions corresponding to the search engine is generated based on the first request field; the first request field and the second request field are different; if satisfied, the first request field is used as the second request field that satisfies the search conditions. Data is retrieved from the cloud database based on the second request field to obtain the first search result.
2. The method as described in claim 1, characterized in that, After obtaining the first search result, it also includes: Based on cloud-native traffic transfer records, retrieve homogeneous data from the first search result; The homogeneous data in the first search result is deduplicated, and the deduplicated first search result is cached in the first-level cache space to obtain the first-level cache data; the first-level cache space includes a first condition sequence table; the first condition sequence table records the mapping relationship between the first request field and the first-level index; the first-level index is used to index the first-level cache data.
3. The method as described in claim 2, characterized in that, The first search result after deduplication is cached in the first-level cache space, resulting in first-level cache data, including: Calculate the capacity of the first-level cache data; Sort and group the fields in the first request field; Sort the first deduplicated search results according to the sorting and grouping of each field; Based on the capacity of the first-level cache data, the sorted first search result is cached in the first-level cache space.
4. The method as described in claim 3, characterized in that, Calculating the capacity of the first-level cache data includes: Query the capacity of the first-level cache space and the capacity of the first search result; The capacity of the cached first-level cache data is calculated based on the capacity of the first-level cache space and the capacity of the first search result; the capacity of the first-level cache data is directly proportional to the capacity of the first-level cache space; the capacity of the first-level cache data is inversely proportional to the capacity of the first search result.
5. The method as described in claim 3, characterized in that, After obtaining the first-level cache data, it also includes: Based on the first request field, data is retrieved from the first-level cache data to obtain the second retrieval result; The second search result is cached in the second-level cache space to obtain the second-level cache data; the second-level cache space includes a second condition sequence table; the second condition sequence table records the mapping relationship between the first request field and the second-level index; the second-level index is used to index the second-level cache data; Feedback is provided on the secondary cache data.
6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: Receive an eviction instruction for L1 cache data; the eviction instruction is sent after the L1 cache data has been modified; Delete the second cache data corresponding to the elimination instruction from the second-level cache space; Remove the first cached data that was not hit within the preset period from the first-level cache space.
7. A cloud computing-based data retrieval device, characterized in that, The device is applied to a data retrieval system, which includes a scheduling service and multiple retrieval engines, each corresponding to a cloud database. The device is applied to each retrieval engine and includes: The acquisition module is used to receive a search request sent by the scheduling service; the search request includes a first request field; the scheduling service acquires the search request sent by the user and sends the search request to the plurality of search engines; The processing module is used to perform condition cleaning on the first request field, wherein condition cleaning refers to converting complex natural language search fields of various forms into standard search fields of the search engine; determine whether the first request field meets the search conditions corresponding to the search engine; the search conditions are generated based on fields of data in the cloud database corresponding to the search engine, representing fields in the cloud database that are allowed to be indexed; the search conditions of the multiple search engines are different; if not, a second request field that meets the search conditions corresponding to the search engine is generated based on the first request field; the first request field and the second request field are different; if they are, the first request field is used as the second request field that meets the search conditions; data is retrieved from the cloud database based on the second request field to obtain a first search result.
8. A cloud computing-based data retrieval system, characterized in that, include: Scheduling services and multiple search engines; each search engine corresponds to a cloud database; For any given search engine, the search criteria are generated based on fields from data in the cloud database that is connected to the search engine, representing the fields in the cloud database that are allowed to be indexed; the search criteria for the multiple search engines are different. The scheduling service is used to obtain the search request sent by the user and send the search request to the plurality of search engines; the search request includes a first request field; For any search engine, the search engine performs condition cleaning on the first request field, wherein condition cleaning refers to converting complex natural language search fields of various forms into standard search fields of the search engine; determining whether the first request field meets the search conditions corresponding to the search engine; if not, generating a second request field that meets the search conditions of the search engine based on the first request field; the first request field and the second request field are different; if they are, using the first request field as the second request field that meets the search conditions; retrieving data from the cloud database based on the second request field to obtain a first search result.
9. A computer device, characterized in that, include: Memory, used to store program instructions; A processor is configured to invoke program instructions stored in the memory and execute the method according to any one of claims 1 to 6.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing a computer to perform the method according to any one of claims 1 to 6.