Data query method and device, computer equipment and computer storage medium
By compressing and encoding the host identity list, smaller query statements are generated, and query plug-in is used to perform query processing, the problem of inefficiency of large-scale ID queries and cross-data source queries in the prior art is solved, and faster query operations and lower network transmission costs are achieved.
Patent Information
- Application Number
- CN202411954147.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-27
- Publication Date
- 2025-04-25
AI Technical Summary
The existing Elasticsearch-based query methods have inefficient problems when handling large-scale ID queries and cross-data source queries, especially in terms of the increase in IO overhead and limitations in data storage location caused by excessive query statements.
The host identity list is compressed and encoded through a pre-built custom plug-in, and smaller query statements are generated, and the pre-built query plug-in is used for query processing to obtain the query results corresponding to the user identity.
The size of the query statement is effectively reduced, network congestion caused by large-scale ID transmission in traditional query methods is avoided, the amount of data transmitted on the network is significantly reduced, the transmission speed of query requests is accelerated, and the efficiency of query operations is improved.
Smart Images

Figure CN119884171A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of query optimization, and in particular to a data query method, device, computer equipment and computer storage medium. Background Art
[0002] In the current era of rapid information development, IT (Information Technology System) has increasingly higher requirements for data storage and query efficiency. As a distributed, highly scalable and highly real-time search and data analysis engine, Elasticsearch has been widely used in various IT systems to achieve efficient data retrieval and analysis functions.
[0003] However, with the rapid growth of data volume and the diversification of query requirements, the current Elasticsearch-based query methods have gradually exposed some significant defects, especially when dealing with large-scale ID queries and cross-data source queries. For example, the current IT system generally uses Elasticsearch's terms (multi-value precise query operation) to query a large number of IDs. This method may be able to cope with a small number of IDs, but when faced with millions or even more IDs, its limitations are clearly revealed. Summary of the invention
[0004] Based on this, it is necessary to provide a data query method, device, computer equipment and computer storage medium that can solve the above-mentioned technical problems.
[0005] In a first aspect, the present application provides a data query method, the method comprising:
[0006] Determine a host identity list based on the pre-acquired user identity and host query conditions;
[0007] When the number of host identifiers in the host identifier list is greater than a preset number, a pre-built custom plug-in is used to compress and encode the host identifier list to obtain a query statement;
[0008] A pre-built query plug-in is used to perform query processing according to the query statement to obtain the query result corresponding to the user identity.
[0009] In one embodiment, the method further comprises:
[0010] When the number of host identities in the host identity list is less than a preset number, a query plug-in is used to perform query processing according to the host identity list to obtain a query result corresponding to the user identity.
[0011] In one embodiment, the pre-built custom plug-in is used to compress and encode the host identity list to obtain a query statement, including:
[0012] The compressed bitmap algorithm in the custom plug-in is used to convert the host identity list into a compressed bitmap format.
[0013] Serializing the compressed bitmap format data to obtain serialized data;
[0014] Encode the serialized data to obtain encoded data;
[0015] Determine the query statement based on the encoded data.
[0016] In one embodiment, determining the query statement according to the encoded data includes:
[0017] Obtain compression encoding parameters in the encoded data;
[0018] The search builder in the search engine is used to embed the compression encoding parameter into the builder statement to obtain the query statement.
[0019] In one embodiment, the pre-built query plug-in performs query processing according to the query statement to obtain the query result corresponding to the user identity, including:
[0020] The query plug-in is used to parse the query statement to obtain parsed data; the parsed data includes the parsed query statement and compression encoding parameters;
[0021] Use the query plug-in to decode and restore the compressed encoding parameters to obtain serialized data;
[0022] Use the query plug-in to deserialize the serialized data to obtain a host identity list;
[0023] The query algorithm preset in the search engine is used to perform query processing according to the parsed query statement and the host identity list to obtain the query result corresponding to the user identity.
[0024] In one embodiment, the query processing is performed based on the parsed query statement and the host identity list to obtain the query result corresponding to the user identity, including:
[0025] Get the query conditions in the parsed query statement;
[0026] Perform data retrieval processing in a preset database according to the query conditions to obtain multiple retrieval data;
[0027] The plurality of search data are screened and processed according to the host identity identifiers in the host identity identifier list to obtain a query result corresponding to the user identity identifier.
[0028] In one embodiment, the method further comprises:
[0029] Obtain business conditions;
[0030] The query results corresponding to the user identity identifier are filtered according to the business conditions to obtain filtered query results.
[0031] In a second aspect, the present application also provides a data query device. The device includes:
[0032] A list determination module, used to determine a host identity list based on pre-acquired user identity and host query conditions;
[0033] A statement determination module, used for compressing and encoding the host identity list using a pre-built custom plug-in to obtain a query statement when the number of host identity identifiers in the host identity list is greater than a preset number;
[0034] The query module is used to use a pre-built query plug-in to perform query processing according to the query statement to obtain the query result corresponding to the user identity.
[0035] In a third aspect, the present application further provides a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method in the first aspect when executing the computer program.
[0036] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the method in the first aspect when executed by a processor.
[0037] In a fifth aspect, the present application further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the method in the first aspect are implemented.
[0038] The above-mentioned data query method, device, computer equipment and computer storage medium determine the host identity list based on the pre-acquired user identity and host query conditions. When the number of host identity in the host identity list is greater than the preset number, a pre-built custom plug-in is used to compress and encode the host identity list to obtain a query statement, and a pre-built query plug-in is used to perform query processing according to the query statement to obtain a query result corresponding to the user identity. The present application compresses and encodes the host identity list through a custom plug-in, which effectively reduces the size of the query statement. In particular, when processing large-scale host identity, it avoids network congestion caused by a large number of ID transmissions in traditional query methods, significantly reduces the amount of data transmitted in the network, and speeds up the transmission speed of query requests, so that query operations can reach the search engine for processing more quickly. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] Figure 1 An application environment diagram of a data query method in an embodiment;
[0040] Figure 2 A schematic diagram of a flow chart of a data query method in an embodiment;
[0041] Figure 3 A schematic diagram of a flow chart of compression and encoding processing in one embodiment;
[0042] Figure 4 A schematic diagram of a flow chart of determining a query statement according to encoded data in one embodiment;
[0043] Figure 5 A schematic diagram of a process for obtaining a query result corresponding to a user identity in one embodiment;
[0044] Figure 6 A schematic diagram of a process for determining a query result corresponding to a user identity in one embodiment;
[0045] Figure 7 is a flow chart of a data query method in another embodiment;
[0046] Figure 8 is a structural block diagram of a data query device in an embodiment;
[0047] Fig. 9 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION
[0048] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0049] First of all, before specifically introducing the technical solution of the embodiments of the present application, the technical background based on the embodiments of the present application is introduced.
[0050] In the current era of rapid development of informatization, IT systems have increasingly higher requirements for data storage and query efficiency. As a distributed, highly scalable and highly real-time search and data analysis engine, Elasticsearch has been widely used in various IT systems to achieve efficient data retrieval and analysis functions. However, with the rapid growth of data volume and the diversification of query requirements, the existing query methods based on Elasticsearch have gradually exposed some significant defects, especially when dealing with large-scale ID queries and cross-data source queries.
[0051] Specifically, the current IT system generally uses the method of directly querying a large number of IDs through the Elasticsearch terms operation. This method may be able to cope with a small number of IDs, but when faced with millions or even more IDs, its limitations are clearly revealed. First, because Elasticsearch query statements do not support effective parameter compression for a large number of IDs, the size of the query statement expands dramatically as the number of IDs increases. For example, when it is necessary to query consecutive IDs from 1 to 50,000, the size of the query statement may be as high as 283KB, which not only increases the burden of network transmission, but also significantly increases IO overhead, thereby affecting the query response speed.
[0052] Second, existing methods require that all queried data must be stored in Elasticsearch, which is often difficult to achieve in practical applications. Data in many IT systems is stored in a decentralized manner. User information, host information, etc. may be stored in relational databases, while time-related data is accumulated in Elasticsearch. This uneven distribution of data makes some data impossible or unsuitable for redundant storage in Elasticsearch, thus limiting the ability to use Elasticsearch alone for queries.
[0053] In summary, the existing Elasticsearch-based query methods have obvious shortcomings when processing large-scale ID queries and cross-data source queries, mainly reflected in the increase in IO overhead caused by large query statements and the limitation of data storage locations. Therefore, there is an urgent need for a query method that can optimize the size of query statements, improve query efficiency, and flexibly adapt to different data storage methods to meet the growing query needs of IT systems.
[0054] Based on this, the present application provides a data query method, apparatus, computer device and computer storage medium, aiming to solve the above technical problems.
[0055] The data query method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The terminal determines the host identity list based on the pre-acquired user identity and host query conditions. When the number of host identities in the host identity list is greater than the preset number, the host identity list is compressed and encoded using a pre-built custom plug-in to obtain a query statement, and a pre-built query plug-in is used to perform query processing according to the query statement to obtain the query result corresponding to the user identity. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones and tablet computers. The server 104 can be implemented with an independent server or a server cluster consisting of multiple servers.
[0056] In an exemplary embodiment, Figure 2 As shown, a data query method is provided, which is applied to Figure 1 The terminal in is used as an example to illustrate, including the following steps:
[0057] S201, determining a host identity list based on pre-acquired user identity and host query conditions.
[0058] Among them, the user identity is used to uniquely identify the information of the system user. For example, in an enterprise information management system, it can be an employee's work number, user name, etc. It is an important basis for subsequent query operations and is used to associate and filter data related to a specific user.
[0059] Host query conditions are conditions for filtering and limiting hosts, such as the host name, IP address, geographical location, operating system type, and other information. These conditions can be used to determine the target host range for querying data from a large number of hosts.
[0060] The host identity list is a list of unique identifiers of hosts that meet the user identity and host query conditions. These identifiers can accurately identify each host in the system, such as the host ID number, etc., for subsequent data query operations on these specific hosts.
[0061] In an embodiment of the present application, when a user logs into the system, the system automatically obtains the user's login credentials (such as a user name and password), verifies the user's identity by comparing it with the user authentication database, and obtains the user identity (such as a user ID). The system provides a graphical query interface, and the user enters the host query conditions on the interface, such as partial keywords of the host name, IP address segments, etc. The system converts the query conditions entered by the user into a query statement format that can be recognized by the database. Using the database query interface, a query request containing the user identity and the converted host query conditions is sent to a relational database. The database executes a query operation, retrieves the host records that meet the conditions from the table storing the host information according to the permissions associated with the user identity and the host query conditions, extracts the host identity (such as the host ID) and forms a host identity list and returns it to the system.
[0062] S202: When the number of host identifiers in the host identifier list is greater than a preset number, a pre-built custom plug-in is used to compress and encode the host identifier list to obtain a query statement.
[0063] Among them, the custom plug-in is a plug-in with specific functions specially developed for Elasticsearch. It is mainly used to compress and encode the integer ID parameter in the query statement (here is the ID in the host identity list) to reduce the size of the query statement, and at the same time implement functions such as filtering algorithms for digital type fields to improve query efficiency and performance.
[0064] In the embodiment of the present application, after the host identity list is determined, the custom plug-in in the system is triggered. The plug-in first loads the RoaringBitmap algorithm library, takes each ID in the host identity list as input, and calls the relevant function of RoaringBitmap to convert it into a data structure in the RoaringBitmap format. In this process, RoaringBitmap will optimize storage according to the distribution characteristics of the ID (such as a continuous ID range) to reduce the space occupied by data.
[0065] The converted RoaringBitmap data is serialized and converted into a byte stream. The serialized byte stream can be easily transmitted and stored in the network while maintaining the integrity of the data.
[0066] Then use the Base64 encoding library to encode the serialized byte stream and convert the binary data into a printable ASCII character sequence. The encoded string is used as a compressed and encoded host identity list. It is combined with the query field name (such as "host_id") and other information according to the format requirements of the Elasticsearch query statement to construct the final query statement. For example, the constructed query statement can be: {"query":{"terms":{"host_id":"<compressed and encoded host ID list>"}}}.
[0067] Another implementation method: When the custom plug-in detects that the number of host identifiers is greater than a preset number (such as 100), it starts a background thread dedicated to compression and encoding processing. In this thread, the host identifier list is first pre-processed to remove possible duplicate IDs to further optimize the data.
[0068] The optimized RoaringBitmap algorithm is used to convert the host identity list into the RoaringBitmap format. In this process, a dynamic memory allocation strategy is used to reasonably allocate memory according to the actual data volume to improve memory usage efficiency. The serialized RoaringBitmap data is segmented and Base64 encoded, and the data is divided into several smaller segments and encoded separately, which can reduce memory usage and avoid memory pressure caused by processing a large amount of data at one time. Finally, the encoded segments are combined into a complete query statement to ensure the stability and accuracy of the query statement during network transmission and Elasticsearch parsing.
[0069] S203, using a pre-built query plug-in to perform query processing according to the query statement to obtain a query result corresponding to the user identity identifier.
[0070] Among them, the query plug-in is a plug-in that can interact with search engines (such as Elasticsearch), perform query operations, and process query results.
[0071] In an embodiment of the present application, after the query plug-in receives the constructed query statement, it first parses the query statement to identify the query type (such as a custom sd_terms query method), query fields (such as host-related fields), and compressed and encoded parameters (i.e., the processed part of the host identity list).
[0072] Based on the parsing results, the query plug-in uses the Java client API (or other applicable programming language clients) provided by Elasticsearch to establish a connection with the Elasticsearch search engine and send the query statement to Elasticsearch through network transmission.
[0073] After receiving the query statement, Elasticsearch executes the query operation and searches for documents matching the query conditions in its distributed index data. The query plug-in waits for Elasticsearch to return the query results, which are returned in a specific data format (such as JSON format).
[0074] The query plug-in receives the original query results returned by Elasticsearch, parses each document in the results, and extracts the relevant field values. According to the pre-set business conditions (such as querying data within the last week, data with a specific business status, etc.), each document is filtered through the written business logic code. For example, it traverses each document in the results, checks whether the timestamp field in the document is within the last week, and whether the business status field meets the requirements. Only documents that meet all business conditions are retained in the final query result set, and finally the filtered results are returned to the user or other modules that call the query.
[0075] Another implementation method: After receiving the query statement, the query plug-in first caches the query statement locally and sends it asynchronously to Elasticsearch for query processing. While waiting for Elasticsearch to return the result, the query plug-in can continue to process other tasks to improve the concurrent processing capability of the system. When Elasticsearch returns the original query result, the query plug-in uses multi-threading technology to start multiple threads to process the result at the same time. Each thread is responsible for parsing a part of the result document and filtering business conditions to improve the processing speed. In the process of filtering the results, the query plug-in uses a memory-mapped file-based method to store and process a large amount of query result data, reducing memory usage and improving data processing efficiency. By mapping the result data to memory, the data on the disk can be accessed and processed as quickly as operating memory data, speeding up the filtering speed. Finally, the query plug-in merges and organizes the results after multi-thread filtering, removes possible duplicate results (if any), and sorts the results according to the sorting rules specified by the user (such as descending time), and then returns the final query results.
[0076] The above-mentioned data query method, device, computer equipment and computer storage medium determine the host identity list based on the pre-acquired user identity and host query conditions. When the number of host identity in the host identity list is greater than the preset number, a pre-built custom plug-in is used to compress and encode the host identity list to obtain a query statement, and a pre-built query plug-in is used to perform query processing according to the query statement to obtain a query result corresponding to the user identity. The present application compresses and encodes the host identity list through a custom plug-in, which effectively reduces the size of the query statement. In particular, when processing large-scale host identity, it avoids network congestion caused by a large number of ID transmissions in traditional query methods, significantly reduces the amount of data transmitted in the network, and speeds up the transmission speed of query requests, so that query operations can reach the search engine for processing more quickly.
[0077] In an exemplary embodiment, based on the above embodiment, the method of the embodiment of the present application further includes:
[0078] When the number of host identities in the host identity list is less than a preset number, a query plug-in is used to perform query processing according to the host identity list to obtain a query result corresponding to the user identity.
[0079] In the embodiment of the present application, after obtaining the host identity list, the terminal immediately counts the number of host identity identifiers in the list. The number of hosts can be determined by traversing the list elements and counting, or by using the built-in function provided by the programming language to obtain the list length. The query plug-in builds a simple query statement structure based on the query requirements and system configuration. Each ID in the host identity list is added to the given position in the query statement template in turn to form a complete query statement.
[0080] The query plug-in uses the connection interface with Elasticsearch (such as the RestHighLevelClient of the Java client) to send the constructed query statement to the Elasticsearch search engine. After receiving the query request, the search engine performs a matching search in its index data according to the conditions in the query statement. Elasticsearch executes the query operation, traverses its stored document data, and finds all documents whose host_id field values match the IDs in the host identity list in the query statement. These matching documents are then returned to the query plug-in as query results. After receiving the original query results returned by Elasticsearch, the query plug-in parses the results. The parsed results are filtered according to pre-set business conditions (such as time range, data status, etc.). The final query results after filtering are returned to the upper-level module or user interface that calls the query function in an appropriate format (such as a list, table, or custom data object) so that users can view and use the query results.
[0081] In the embodiment of the present application, when the number of host identifiers is less than a preset number, a query processing method based directly on the host identifier list is adopted, which avoids unnecessary complex compression and encoding operations and reduces the intermediate links of query processing. This direct query method can quickly construct a query statement and submit it to the search engine, so that the query request can reach the search engine and be processed more quickly, shortening the overall response time of the query and improving the efficiency of the system in processing small-scale host-related queries. For example, in a small enterprise network management system, when an administrator queries the information of a few specific hosts, the query results can be obtained quickly, improving management efficiency.
[0082] In an exemplary embodiment, based on the above embodiment, see Figure 3 The embodiment of the present application involves using a pre-built custom plug-in to compress and encode a host identity list to obtain a query statement, including:
[0083] S301, using a compressed bitmap algorithm in a custom plug-in to perform format conversion processing on a host identity list to obtain compressed bitmap format data.
[0084] In the embodiment of the present application, the custom plug-in can load the RoaringBitmap algorithm library, obtain the host identity list, and then traverse each host identity in the list (assuming it is an integer type ID). For each ID, the RoaringBitmap algorithm determines the container to which it belongs based on its value (RoaringBitmap divides the integer range into multiple containers for management).
[0085] If the ID belongs to a smaller numerical range (such as 0 to 65535), it is added to the ShortContainer, which uses 16 bits to represent integers. For consecutive IDs, it compresses the storage by recording the starting ID and the number of consecutive IDs. For example, if there are consecutive IDs from 100 to 110, it will record the starting ID as 100 and the consecutive number as 11.
[0086] If the ID belongs to a larger numerical range, it is added to the BitmapContainer, which uses a bitmap to indicate the presence or absence of an integer. Each bit corresponds to an integer, and the value of the bit (0 or 1) is set to indicate whether the integer is in the set. When all host identifiers are processed and added to the corresponding container, the format conversion is completed to obtain compressed bitmap format data, namely the RoaringBitmap object, which represents the compressed representation of the host identifier list.
[0087] S302, serializing the compressed bitmap format data to obtain serialized data.
[0088] In the embodiment of the present application, the RoaringBitmap object is converted into a byte stream. In the serialization process, each attribute of the RoaringBitmap object (such as the container type, the data in the container, etc.) is converted into a byte sequence in turn according to the standard protocol of Java object serialization.
[0089] For different containers in RoaringBitmap (such as ShortContainer and BitmapContainer), serialization is performed according to their internal structures. For example, for ShortContainer, serialize its starting ID, number of consecutive IDs and other information; for BitmapContainer, serialize each bit information in the bitmap. After serialization, a byte array is obtained, which is the serialized data. It fully represents the content of the compressed bitmap format data and can be easily stored or transmitted.
[0090] S303, encoding the serialized data to obtain encoded data.
[0091] In the embodiment of the present application, the terminal can use a Base64 encoding library (such as java.util.Base64) to encode the serialized byte array as input. During the encoding process, each byte in the byte array is converted into a corresponding ASCII character according to the Base64 encoding rules.
[0092] Base64 encoding converts three bytes of data into four printable ASCII characters. For less than three bytes, padding characters (such as =) are added to ensure that the encoded string length is a multiple of 4. For example, a byte array [0x12, 0x34, 0x56] may be encoded with Base64 to obtain the string "MTIzNDU2".
[0093] After the encoding is completed, a string containing Base64-encoded data is obtained. This string is the encoded data, which has better compatibility and readability and is suitable for use in network transmission and storage.
[0094] S304: Determine a query statement according to the encoded data.
[0095] In the embodiment of the present application, a query statement is constructed according to the interaction protocol between the query plug-in and the search engine (such as Elasticsearch) and the query grammar rules. For example, for Elasticsearch, the query statement may be a string in JSON format, including information such as the query type and the query field.
[0096] Embed the encoded data (Base64 encoded string) into the corresponding position of the query statement. Assuming the query field is host_id, the constructed query statement may be similar to {"query":{"terms":{"host_id":"<encoded data>"}}}, where the <encoded data> part is replaced with the actual Base64 encoded string. Perform syntax check and optimization on the constructed query statement to ensure that it meets the syntax requirements of the search engine and can be executed efficiently.
[0097] Another implementation method: The query plug-in fills the encoded data into the template according to the pre-defined query template. The query template can be customized according to different query requirements and scenarios. For example, for complex queries containing multiple query conditions (in addition to the host identity, there are also time ranges, business types, etc.), the template reserves corresponding positions for filling in data with different conditions.
[0098] When determining the query statement, the query statement is hashed to generate a unique hash value, taking into account the cache mechanism of the search engine. The hash value is stored in the local cache together with the query statement (if there is no query statement with the same hash value in the cache), so that the query statement can be directly obtained from the cache for subsequent identical queries, which improves the query building speed and reduces unnecessary repeated building operations.
[0099] The embodiment of the present application converts the format of the host identity list through a compressed bitmap algorithm, and can store the host identity data that may originally occupy a large amount of space in a more compact bitmap form, greatly reducing the storage space of the data. During the data transmission process, whether it is transmitted between internal components of the system or network transmission from the client to the server, a smaller amount of data means a lower transmission bandwidth requirement, reducing the network transmission cost, which is particularly suitable for frequent query operations in large-scale distributed systems, reducing the occupation of network resources by data transmission, and improving the overall transmission efficiency of the system.
[0100] In an exemplary embodiment, based on the above embodiment, see Figure 4 The embodiment of the present application relates to a process of determining a query statement according to encoded data, comprising the following steps:
[0101] S401, obtaining compression encoding parameters in the encoded data.
[0102] The compression encoding parameter is a key part of the data obtained after a series of processing (such as RoaringBitmap compression and Base64 encoding). It contains the compressed and encoded information of the host identity list and is an important parameter used to accurately specify the query conditions when constructing a query statement.
[0103] In the embodiment of the present application, after receiving the encoded data (assuming that it is in the form of a string encoded by Base64), the format and structure of the encoded data are first determined. According to the predefined encoding rules, the starting and ending positions representing the compression encoding parameters are found. For example, if the encoding rules stipulate that the first 10 characters are metadata (such as an encoding type identifier, etc.), then the part from the 11th character to the end of the string is the compression encoding parameter.
[0104] Use the string processing function provided by the programming language (such as the slice operation in Python) to extract the compression encoding parameter part. For example, if the encoded data is "metaDataPart<compression encoding parameter part>", then the <compression encoding parameter part> is obtained through the slice operation data[10:].
[0105] Perform necessary verification and preprocessing on the extracted compression encoding parameters. Check the integrity of the parameters, for example, verify whether their length meets expectations, whether they contain specific checksum information (if any), etc. If any damage or incompleteness of the parameters is found, take appropriate error handling measures, such as recording error logs and trying to re-acquire data or prompting the user of data errors.
[0106] S402, using a search builder in a search engine to embed compression encoding parameters into a builder statement to obtain a query statement.
[0107] Among them, the search builder is a tool or component provided by a search engine (such as Elasticsearch) for building query statements. It provides a series of methods and interfaces, allowing developers to build complex query statements according to the grammatical rules of the search engine.
[0108] In the embodiment of the present application, a corresponding search builder object is instantiated according to the type of search engine (such as Elasticsearch). For example, in Java, using the Java client library of Elasticsearch, a query builder object for the host_id field is created through newTermsQueryBuilder("host_id"), where host_id is a query field related to the host identity.
[0109] Use the method of the search builder object to embed the extracted compression encoding parameter into the builder statement. For the TermsQueryBuilder object created above, call its value method and pass in the compression encoding parameter, such as queryBuilder.value(compressedEncodingParam), so that the parameter is set as the value of the query condition.
[0110] Further improve the query builder statement according to the query requirements. For example, if you need to add other query conditions (such as time range restrictions), you can continue to call other methods of the builder object to add conditions. The conditions are as follows:
[0111] queryBuilder.filter(RangeQueryBuilder("timestamp").gte(startTime).lte(endTime)). "timestamp" is the time field, startTime and endTime are the set time ranges. In this way, a complex query statement containing host identity conditions and time range conditions is constructed.
[0112] Finally, the constructed query statement is converted into the final query statement format (such as a JSON format string) by calling the build method of the builder object so as to be submitted to the search engine for query operations.
[0113] The embodiment of the present application ensures the accuracy of the query condition by accurately obtaining the compression encoding parameters in the encoded data. When constructing a query statement, only valid and processed parameters are used to avoid query deviations caused by incorrect or incomplete parameters. At the same time, the search builder provided by the search engine is used to construct the query statement, which follows the grammatical rules and best practices of the search engine and reduces the grammatical errors that may occur when manually constructing the query statement. This method improves the efficiency of query construction, enables query statements to be generated quickly and accurately, reduces the time cost of developing and maintaining query functions, improves the development efficiency and stability of the system, and is particularly suitable for situations where different query statements need to be frequently constructed in complex query scenarios.
[0114] In an exemplary embodiment, based on the above embodiment, see Figure 5 The embodiment of the present application involves using a pre-built query plug-in to perform query processing according to a query statement to obtain a query result corresponding to a user identity, including:
[0115] S501, using a query plug-in to parse the query statement to obtain parsed data.
[0116] The parsed data includes the parsed query statement and compression encoding parameters.
[0117] The query plug-in of the embodiment of the present application uses a built-in syntax parser to parse the query statement. The syntax parser decomposes the query statement into a syntax tree structure according to the query syntax rules of the search engine.
[0118] During the parsing process, the compression encoding parameters are extracted at the same time. By traversing the syntax tree, the node related to the host identity is found and extracted as the compression encoding parameter. The location of the compression encoding parameter can be determined by string matching or according to the attributes of the syntax tree node.
[0119] The parsed query statement and the extracted compression encoding parameters are combined into a parsed data structure to facilitate the processing of subsequent steps.
[0120] S502, using a query plug-in to decode and restore the compressed encoding parameters to obtain serialized data.
[0121] In the embodiment of the present application, the compression encoding parameter is decoded using a decoding algorithm corresponding to the encoding process. If the compression encoding parameter is a Base64-encoded string, the query plug-in calls the Base64 decoding function to convert it into a byte array.
[0122] During the decoding process, error detection and processing are performed. For example, check whether the length of the decoded byte array is reasonable, whether it contains invalid characters, etc. If a decoding error is found, take appropriate measures according to the error type, such as trying to use an alternative decoding method, recording an error log, and notifying the system administrator. The decoded byte array is used as serialized data and prepared for the next step of deserialization.
[0123] S503, using the query plug-in to deserialize the serialized data to obtain a host identity list.
[0124] In the embodiment of the present application, the serialized data is deserialized using a deserialization library according to a predetermined data structure definition and serialization rules. If the serialized data is in the form of a byte stream, the deserialization library converts the byte stream into a data structure of a host identity list in the order and format of the previous serialization.
[0125] During the deserialization process, possible version compatibility issues are handled. If the data structure changes between versions, the deserialization library will handle it according to the pre-set version compatibility strategy, for example, ignoring incompatible fields or performing field mapping conversion to ensure that the host identity list can be deserialized correctly.
[0126] Perform integrity check on the host identity list obtained by deserialization. Verify whether the host identity in the list is valid, whether there are duplications or missing, etc. If problems are found, repair the data or take corresponding actions according to the system policy, such as discarding invalid identities or trying to re-acquire data.
[0127] S504, using a query algorithm preset in the search engine, performing query processing according to the parsed query statement and the host identity list, and obtaining a query result corresponding to the user identity.
[0128] In the embodiment of the present application, the search engine determines the query type and query conditions according to the parsed query statement. For example, if the query statement is a terms query type, the search engine will recognize that an exact match query needs to be performed according to the host identity list.
[0129] Using its internal index structure (such as the inverted index), it can quickly locate documents related to the identifiers in the host identifier list. The inverted index associates each host identifier with a list of documents containing the identifier. By quickly finding these associations, it can quickly find a collection of documents that may meet the query conditions. The found document collection is further filtered according to other conditions in the query statement. For example, if the query statement contains a time range condition, the search engine will check whether the time field in the document is within the specified range. Only documents that meet all conditions will be retained in the final query results.
[0130] The filtered documents are sorted and packaged according to a predetermined format and returned to the query plug-in as the query result corresponding to the user identity.
[0131] The embodiment of the present application can accurately understand the query intent, including key information such as query type, query conditions, and compression encoding parameters, through accurate parsing of query statements, and provides a correct basis for subsequent query processing. In the decoding, restoration and deserialization process, reverse operations are strictly performed in accordance with the encoding rules to ensure that the host identity list can be accurately restored and query condition errors caused by data conversion errors are avoided. Finally, the query algorithm preset by the search engine is used to perform query processing based on accurate query statements and host identity lists, which can accurately filter out qualified results in the data set, ensure that the query results are highly matched with the actual needs of users, improve the accuracy and reliability of query results, enable users to obtain truly useful information, and reduce problems such as decision-making errors or low work efficiency caused by query errors.
[0132] In an exemplary embodiment, based on the above embodiment, see Figure 6 The embodiment of the present application relates to a process of performing query processing according to the parsed query statement and the host identity list to obtain a query result corresponding to the user identity, including the following steps:
[0133] S601, obtaining query conditions in the parsed query statement.
[0134] In an embodiment of the present application, the parsed query statement usually exists in a certain data structure (such as JSON format or a custom parsing tree structure). The query plug-in uses a traversal algorithm to traverse this data structure to find nodes or attributes representing query conditions. For example, for a query statement in JSON format {"query": {"terms": {"host_id": ["host1", "host2"]}, "range": {"timestamp": {"gte": "2023-01-01", "lte": "2023-02-01"}}}}, first find the terms and range nodes under the query node, and extract the host identity list (["host1", "host2"]) as a host-based query condition, and the time range ("2023-01-01" to "2023-02-01") as a time condition.
[0135] Convert the extracted query conditions into a format suitable for query operations in the preset database. If the preset database is a relational database, you may need to convert the query conditions into the WHERE clause part of the SQL statement. For example, convert the host identity condition into host_id IN ('host1', 'host2') and the time range condition into timestamp BETWEEN '2023-01-01' AND '2023-02-01' to facilitate subsequent database query operations.
[0136] S602, performing data retrieval processing in a preset database according to the query conditions to obtain a plurality of retrieval data.
[0137] In the embodiment of the present application, if the preset database is a relational database, the query plug-in uses the database connection library to establish a connection with the database, and constructs a complete query statement according to the converted query conditions.
[0138] Execute the query statement through the database connection to obtain the retrieved data. The database engine scans and matches the data in the table according to the query statement, finds the record rows that meet the conditions, and returns these records as a result set to the query plug-in. After receiving the result set, the query plug-in can convert it into a format suitable for subsequent processing as needed.
[0139] S603, screening and processing the plurality of search data according to the host identity in the host identity list to obtain a query result corresponding to the user identity.
[0140] In an embodiment of the present application, the terminal traverses each record in the retrieved data and checks whether the host identity field in the record matches the identifier in the host identity list. For the result set returned by the relational database, the value of the host identity field is obtained row by row and compared with each identifier in the host identity list. For example, if the host identity list is ["host1", "host2"], when the host identity field value of a record traversed is host1, it is determined that the record meets the host identity condition.
[0141] For records that meet the host identity condition, further filter them according to other query conditions. If the query condition includes business status requirements, check whether the business status field in the record meets the specified conditions. Only records that meet both the host identity condition and all other query conditions are retained in the final query results.
[0142] The embodiment of the present application can accurately guide the data retrieval and screening process by accurately obtaining the query conditions, ensuring that data highly relevant to user needs is obtained from the preset database. In the data retrieval process, appropriate query methods are used according to different types of preset databases (relational or non-relational), thereby improving the accuracy and efficiency of the retrieval. Finally, the retrieved data is strictly screened according to the host identity list, and irrelevant data is further excluded, so that the query results finally returned fully meet the query requirements corresponding to the user identity, providing users with accurate and targeted data, reducing the time and energy consumption of users in processing large amounts of irrelevant data, and improving work efficiency and decision-making accuracy. It is especially suitable for business scenarios with extremely high requirements for data accuracy, such as financial data analysis, medical information query, etc.
[0143] In an exemplary embodiment, based on the above embodiment, see Figure 7 , the method according to the embodiment of the present application further includes the following steps:
[0144] S701, obtaining business conditions.
[0145] In the embodiment of the present application, a user interface may be provided, and the user may specify business conditions on the interface when performing a query operation. For example, in a data query Web application, there are special input boxes and drop-down menus for users to select a time range (such as a selection box for the start date and the end date), business status (such as options such as "completed", "unfinished", and "in progress" in the drop-down menu), and event type (such as selecting "login event", "transaction event", "data update event", etc. through a multiple-selection box). After the user completes the setting of the business conditions on the interface, click the query button, and the system collects the business condition information set by the user.
[0146] The terminal converts the collected business condition information into a format suitable for processing in the program, usually a data structure containing various business condition attributes and corresponding values.
[0147] S702, filtering the query results corresponding to the user identity identifier according to the business conditions to obtain filtered query results.
[0148] In the embodiment of the present application, after obtaining the query result corresponding to the business condition and the user identity, each record in the query result is traversed, and for each record, each field thereof is checked to see whether it satisfies the business condition.
[0149] For records that do not meet any business conditions, they are removed from the query results. Only records that meet all business conditions are retained in the final filtered query results. Finally, the filtered query results are formatted in a suitable format.
[0150] By obtaining business conditions, the embodiment of the present application enables the system to understand the precise needs of users in specific business scenarios, thereby filtering out data closely related to the business in the query results. The query results are strictly screened according to the business conditions, and data that is irrelevant or inconsistent with business logic is removed, ensuring that the filtered query results ultimately returned to the user are highly accurate and can directly support the user's business decisions, analysis or operations. This eliminates the need for users to manually filter useful information from a large number of original query results, saving time and energy, improving work efficiency, and also reducing the risk of business decisions caused by the use of incorrect or irrelevant data. It is particularly suitable for scenarios in complex business environments that have extremely high requirements for data accuracy and pertinence, such as corporate financial analysis, production management decisions, etc.
[0151] In an exemplary embodiment, based on the above embodiment, the method of the embodiment of the present application further includes the following steps:
[0152] Step 1: Based on the pre-acquired user identity and host query conditions, determine the host identity list;
[0153] Step 2: When the number of host identifiers in the host identifier list is less than a preset number, a query plug-in is used to perform query processing according to the host identifier list to obtain a query result corresponding to the user identifier;
[0154] Step 3: When the number of host identifiers in the host identifier list is greater than a preset number, the host identifier list is converted into a format using a compressed bitmap algorithm in a custom plug-in to obtain compressed bitmap format data; the compressed bitmap format data is serialized to obtain serialized data; and the serialized data is encoded to obtain encoded data;
[0155] Step 4: Obtain compression encoding parameters in the encoded data; use the search builder in the search engine to embed the compression encoding parameters into the builder statement to obtain a query statement;
[0156] Step 5: Use the query plug-in to parse the query statement to obtain parsed data; the parsed data includes the parsed query statement and the compressed encoding parameters; use the query plug-in to decode and restore the compressed encoding parameters to obtain serialized data; use the query plug-in to deserialize the serialized data to obtain a host identity list;
[0157] Step 6: Using a query algorithm preset in the search engine, obtain query conditions in the parsed query statement; perform data retrieval processing in a preset database according to the query conditions to obtain multiple retrieval data; filter the multiple retrieval data according to the host identity in the host identity list to obtain a query result corresponding to the user identity;
[0158] Step 7: Obtain business conditions; filter the query results corresponding to the user identity according to the business conditions to obtain filtered query results.
[0159] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.
[0160] Based on the same inventive concept, the embodiment of the present application also provides a data query device for implementing the data query method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more data query device embodiments provided below can refer to the limitations on the data query method above, and will not be repeated here.
[0161] In one embodiment, Figure 8 As shown, a data query device 800 is provided, comprising:
[0162] A list determination module 801 is used to determine a host identity list based on a pre-acquired user identity and a host query condition;
[0163] The statement determination module 802 is used to compress and encode the host identity list using a pre-built custom plug-in to obtain a query statement when the number of host identity identifiers in the host identity list is greater than a preset number;
[0164] The query module 803 is used to use a pre-built query plug-in to perform query processing according to the query statement to obtain the query result corresponding to the user identity.
[0165] In one embodiment, the above device further comprises:
[0166] The query result determination module is used to use the query plug-in to perform query processing according to the host identity list to obtain the query result corresponding to the user identity when the number of host identity identifiers in the host identity list is less than a preset number.
[0167] In one embodiment, the statement determination module includes:
[0168] A format conversion unit, used to perform format conversion processing on the host identity list by using a compressed bitmap algorithm in a custom plug-in to obtain compressed bitmap format data;
[0169] A serialization unit, used for serializing the compressed bitmap format data to obtain serialized data;
[0170] The encoding unit is used to encode the serialized data to obtain encoded data;
[0171] The statement determination unit is used to determine the query statement according to the encoded data.
[0172] In one embodiment, the sentence determination unit includes:
[0173] A parameter acquisition subunit, used to acquire compression encoding parameters in the encoded data;
[0174] The statement determination subunit is used to embed the compression encoding parameter into the builder statement by using the search builder in the search engine to obtain the query statement.
[0175] In one embodiment, the query module includes:
[0176] A parsing unit, used to parse the query statement using the query plug-in to obtain parsed data; the parsed data includes the parsed query statement and compression encoding parameters;
[0177] A restoration unit, used for decoding and restoring the compressed encoding parameters using a query plug-in to obtain serialized data;
[0178] A deserialization unit, used to perform deserialization processing on serialized data using a query plug-in to obtain a host identity list;
[0179] The query unit is used to use the query algorithm preset in the search engine to perform query processing according to the parsed query statement and the host identity list to obtain the query result corresponding to the user identity.
[0180] In one embodiment, the query unit includes:
[0181] The condition acquisition subunit is used to acquire the query condition in the parsed query statement;
[0182] The retrieval subunit is used to perform data retrieval processing in a preset database according to the query conditions to obtain multiple retrieval data;
[0183] The data screening subunit is used to screen the multiple search data according to the host identity in the host identity list to obtain the query result corresponding to the user identity.
[0184] In one embodiment, the above device further comprises:
[0185] Business condition acquisition module, used to acquire business conditions;
[0186] The screening module is used to screen the query results corresponding to the user identity according to the business conditions to obtain the screened query results.
[0187] Each module in the above data query device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.
[0188] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig. 9As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be realized through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a data query method is realized. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.
[0189] Those skilled in the art will understand that Fig. 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0190] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:
[0191] Determine a host identity list based on the pre-acquired user identity and host query conditions;
[0192] When the number of host identifiers in the host identifier list is greater than a preset number, a pre-built custom plug-in is used to compress and encode the host identifier list to obtain a query statement;
[0193] A pre-built query plug-in is used to perform query processing according to the query statement to obtain the query result corresponding to the user identity.
[0194] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:
[0195] When the number of host identities in the host identity list is less than a preset number, a query plug-in is used to perform query processing according to the host identity list to obtain a query result corresponding to the user identity.
[0196] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:
[0197] The compressed bitmap algorithm in the custom plug-in is used to convert the host identity list into a compressed bitmap format.
[0198] Serializing the compressed bitmap format data to obtain serialized data;
[0199] Encode the serialized data to obtain encoded data;
[0200] Determine the query statement based on the encoded data.
[0201] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:
[0202] Obtain compression encoding parameters in the encoded data;
[0203] The search builder in the search engine is used to embed the compression encoding parameter into the builder statement to obtain the query statement.
[0204] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:
[0205] The query plug-in is used to parse the query statement to obtain parsed data; the parsed data includes the parsed query statement and compression encoding parameters;
[0206] Use the query plug-in to decode and restore the compressed encoding parameters to obtain serialized data;
[0207] Use the query plug-in to deserialize the serialized data to obtain a host identity list;
[0208] The query algorithm preset in the search engine is used to perform query processing according to the parsed query statement and the host identity list to obtain the query result corresponding to the user identity.
[0209] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:
[0210] Get the query conditions in the parsed query statement;
[0211] Perform data retrieval processing in a preset database according to the query conditions to obtain multiple retrieval data;
[0212] The plurality of search data are screened and processed according to the host identity identifiers in the host identity identifier list to obtain a query result corresponding to the user identity identifier.
[0213] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:
[0214] Obtain business conditions;
[0215] The query results corresponding to the user identity identifier are filtered according to the business conditions to obtain filtered query results.
[0216] According to some embodiments of the present application, a computer program product is also provided, and when the computer program is executed by a processor, the above method can be implemented. The computer program product includes one or more computer instructions. When these computer instructions are loaded and executed on a computer, part or all of the above method can be implemented in whole or in part according to the process or function described in the embodiment of the present application.
[0217] According to some embodiments of the present application, there is also provided a non-transitory computer-readable storage medium including instructions, such as a memory including instructions, and the above instructions can be executed by a processor of an electronic device to complete the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.
[0218] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.
[0219] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.
[0220] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0221] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.
Claims
1. A data query method, characterized in that: The method comprises: Determine a host identity list based on the pre-acquired user identity and host query conditions; When the number of host identifiers in the host identifier list is greater than a preset number, a pre-built custom plug-in is used to compress and encode the host identifier list to obtain a query statement; A pre-built query plug-in is used to perform query processing according to the query statement to obtain a query result corresponding to the user identity identifier.
2. The method according to claim 1, characterized in that: The method further comprises: When the number of host identities in the host identity list is less than the preset number, the query plug-in is used to perform query processing according to the host identity list to obtain a query result corresponding to the user identity.
3. The method according to claim 1, characterized in that The pre-built custom plug-in is used to compress and encode the host identity list to obtain a query statement, including: Using the compressed bitmap algorithm in the custom plug-in to perform format conversion processing on the host identity list to obtain compressed bitmap format data; Performing serialization processing on the compressed bitmap format data to obtain serialized data; Encoding the serialized data to obtain encoded data; The query statement is determined according to the encoded data.
4. The method according to claim 3, characterized in that Determining the query statement according to the encoded data includes: Obtaining compression encoding parameters in the encoded data; The compression encoding parameter is embedded into a builder statement using a search builder in a search engine to obtain the query statement.
5. The method according to claim 4, characterized in that The pre-built query plug-in is used to perform query processing according to the query statement to obtain a query result corresponding to the user identity, including: The query plug-in is used to parse the query statement to obtain parsed data; the parsed data includes the parsed query statement and the compression encoding parameters; The query plug-in is used to decode and restore the compressed encoding parameters to obtain the serialized data; Deserializing the serialized data using the query plug-in to obtain the host identity list; A query algorithm preset in the search engine is used to perform query processing according to the parsed query statement and the host identity identifier list to obtain a query result corresponding to the user identity identifier.
6. The method according to claim 5, characterized in that The query processing is performed according to the parsed query statement and the host identity list to obtain a query result corresponding to the user identity, including: Obtaining query conditions in the parsed query statement; Perform data retrieval processing in a preset database according to the query conditions to obtain multiple retrieval data; The plurality of search data are screened according to the host identity identifiers in the host identity identifier list to obtain a query result corresponding to the user identity identifier.
7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: Obtain business conditions; The query results corresponding to the user identity identifier are filtered according to the business conditions to obtain filtered query results.
8. A data query device, characterized in that: The device comprises: A list determination module, used to determine a host identity list based on pre-acquired user identity and host query conditions; A statement determination module, used for compressing and encoding the host identity list using a pre-built custom plug-in to obtain a query statement when the number of host identity identifiers in the host identity list is greater than a preset number; The query module is used to use a pre-built query plug-in to perform query processing according to the query statement to obtain a query result corresponding to the user identity.
9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.