Data retrieval method and device, equipment, storage medium and computer program product

Abstract

The invention discloses a data retrieval method and device, equipment, a storage medium and a computer program product, and relates to the technical field of data retrieve.The method comprises the steps that word segmentation processing is conducted on long text data, and an inverted index corresponding to the long text data is generated according to a word segmentation result; converting a business rule contained in the expert rule model into a rule element, wherein the rule element represents a rule for searching the business rule in the long text data; and generating a queried statement according to the rule element, and retrieving the long text data based on the queried statement and the inverted index. According to the method, the word segmentation result of the long text data is structured through the inverted index, and meanwhile, the queriable statement is generated according to the rule element converted from the business rule, so that abstract expression of complex logic is realized to enhance manageability and reusability of the business rule; therefore, accurate retrieval can be carried out in massive long text data in combination with the inverted index and the queried statements.

Description

Technical Field

[0001] The present application relates to the field of data retrieval technology, and in particular to a data retrieval method, apparatus, device, storage medium, and computer program product. Background Art

[0002] In the actual business processes of the financial industry, customer service centers need to build a large number of expert rule models in aspects such as anomaly detection, operation assistance, and service quality inspection. Such models need to be retrieved under massive long text data.

[0003] However, traditional text retrieval technology mainly relies on underlying data structures and algorithms. Whether it is the NFA (Non-Deterministic Finite Automaton) of regular expressions, the ANN (Approximate Nearest Neighbor) of vector databases, or the B+Tree (Balanced Multi-way Search Tree) of relational databases, none of them can achieve accurate retrieval in massive long text data. Summary of the Invention

[0004] The main purpose of this application is to provide a data retrieval method, device, equipment, storage medium and computer program product, aiming to solve the technical problem that the existing technology cannot achieve accurate retrieval in massive long text data.

[0005] To achieve the above objectives, the present application provides a data retrieval method, which comprises the following steps:

[0006] Perform word segmentation on the long text data, and generate an inverted index corresponding to the long text data based on the word segmentation result;

[0007] Converting the business rules included in the expert rule model into rule elements, wherein the rule elements represent rules for performing retrieval of the business rules in the long text data;

[0008] A queryable statement is generated according to the rule element, and the long text data is retrieved based on the queryable statement and the inverted index.

[0009] In one embodiment, the step of performing word segmentation on the long text data and generating an inverted index corresponding to the long text data according to the word segmentation result includes:

[0010] Perform word segmentation on the long text data in character units, and determine basic information corresponding to a single character in the long text data based on the word segmentation result, the basic information including document ID, offset, and number of occurrences;

[0011] A triplet of the single character is constructed according to the basic information, and an inverted index corresponding to the long text data is generated according to the triplet of all characters in the long text data, wherein the triplet represents a positioning constraint of the single character in the long text data.

[0012] In one embodiment, the step of converting the business rules included in the expert rule model into rule elements includes:

[0013] Decomposing the business rules contained in the expert rule model to obtain domain keywords and target rankings and target distance tolerances corresponding to the domain keywords;

[0014] A rule element corresponding to the business rule is determined according to the domain keyword, the target ranking, and the target distance tolerance.

[0015] In one embodiment, after the step of determining the rule element corresponding to the business rule according to the domain keyword, the target ranking, and the target distance tolerance, the method further includes:

[0016] Setting an identification ID for the first rule element, and establishing a mapping relationship between the first parsing result corresponding to the first rule element and the identification ID;

[0017] When traversing the business rules included in the expert rule model, if a second rule element that is the same as the first rule element is traversed, the first parsing result is used as the parsing result of the second rule element based on the mapping relationship.

[0018] In one embodiment, the step of generating a queryable statement according to the rule element and searching the long text data based on the queryable statement and the inverted index includes:

[0019] Decomposing the domain keyword in the rule element into a plurality of keyword elements, and binding constraints on the keyword elements to obtain a basic query unit;

[0020] A queryable statement is generated according to the basic query units corresponding to all the field keywords, the target ranking, and the target distance tolerance, and the long text data is retrieved based on the queryable statement and the inverted index.

[0021] In one embodiment, the data retrieval method further includes:

[0022] When receiving a retrieval task for the long text data, obtaining the CPU utilization rate and thread pool queue backlog of the currently available instance;

[0023] A target instance is selected from the currently available instances according to the CPU utilization and the thread pool queue backlog, and the target instance is scheduled to process the retrieval task.

[0024] In addition, to achieve the above-mentioned purpose, the present application also proposes a data retrieval device, which includes:

[0025] An index generation module is used to perform word segmentation on long text data and generate an inverted index corresponding to the long text data based on the word segmentation results;

[0026] A rule parsing module, configured to convert the business rules contained in the expert rule model into rule elements, wherein the rule elements represent rules for performing retrieval of the business rules in the long text data;

[0027] A data retrieval module is used to generate a queryable statement according to the rule element, and retrieve the long text data based on the queryable statement and the inverted index.

[0028] In addition, to achieve the above-mentioned purpose, the present application also proposes a data retrieval device, which includes: a memory, a processor, and a data retrieval program stored on the memory and executable on the processor, wherein the data retrieval program is configured to implement the steps of the data retrieval method described above.

[0029] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium and stores a data retrieval program. When the data retrieval program is executed by a processor, the steps of the data retrieval method described above are implemented.

[0030] In addition, to achieve the above-mentioned object, the present invention further provides a computer program product, which includes a data retrieval program. When the data retrieval program is executed by a processor, the steps of the data retrieval method described above are implemented.

[0031] The present application performs word segmentation processing on long text data, generates an inverted index corresponding to the long text data based on the word segmentation results; converts the business rules contained in the expert rule model into rule elements, wherein the rule elements represent the rules for searching the long text data by the business rules; generates queryable statements based on the rule elements, and searches the long text data based on the queryable statements and the inverted index. The above method of the present application structures the word segmentation results of the long text data through an inverted index, and generates queryable statements based on the rule elements converted from the business rules, thereby realizing the abstract expression of complex logic to enhance the manageability and reusability of the business rules, thereby being able to combine the inverted index with the queryable statements to accurately search in massive long text data. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0033] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0034] Figure 1 This is a flow chart of the first embodiment of the data retrieval method of this application;

[0035] Figure 2 This is an example diagram of converting business rules into rule elements in the data retrieval method of this application;

[0036] Figure 3 This is a flow chart of the second embodiment of the data retrieval method of this application;

[0037] Figure 4 This is a flow chart of the third embodiment of the data retrieval method of the present application;

[0038] Figure 5 This is a schematic diagram of the search process of the data search method of this application;

[0039] Figure 6 Schematic diagram of the multi-instance adaptive dynamic load scheduling mechanism of the data retrieval method of this application;

[0040] Figure 7 This is a structural block diagram of the first embodiment of the data retrieval device of the present application;

[0041] Figure 8 This is a structural diagram of a data retrieval device in the hardware operating environment involved in the embodiment of the present application.

[0042] The realization of the objectives, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0043] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not intended to limit the present application.

[0044] It should be noted that the execution subject of the embodiments of the present application can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, mobile phone, etc., or an electronic device capable of performing the above functions, such as the above-mentioned data retrieval device. The following embodiments are described below using the data retrieval device as an example.

[0045] The present application embodiment provides a data retrieval method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the data retrieval method of this application.

[0046] In this embodiment, the data retrieval method includes the following steps:

[0047] Step S10: performing word segmentation processing on the long text data, and generating an inverted index corresponding to the long text data according to the word segmentation result.

[0048] It should be noted that the aforementioned long text data represents text data containing multiple keywords, complex distance and position constraints, and diverse natural language expressions. For example, long text data can be a detailed customer service conversation record, including information such as the customer's question, the customer service representative's response, operation steps, and timestamps. This information may contain multiple keywords (such as "account balance," "transaction record," "problem resolution," etc.), and these keywords may be subject to complex distance and position constraints.

[0049] It should be understood that in order to solve the problem of poor performance of traditional word segmenters in domain keywords, this embodiment is based on the API (Application Programming Interface) exposed by Lucene (a widely used information retrieval library), and divides Chinese and English characters and punctuation marks into single characters as keywords, cleverly avoiding the problem of poor performance of traditional word segmenters under domain dictionaries. The word segmentation processing method proposed in this embodiment means breaking down the text into the most basic character units instead of relying on predefined vocabulary or rules. This helps to capture domain-specific vocabulary or phrases that traditional word segmenters may ignore, because even if these vocabulary or phrases are not in the dictionary of traditional word segmenters, they can be divided into single characters and processed.

[0050] In a specific implementation, the long text data may be segmented into a number of character units according to the word segmentation result, and then an inverted index corresponding to the long text data may be constructed based on these character units.

[0051] Step S20: converting the business rules included in the expert rule model into rule elements, wherein the rule elements represent rules for searching the long text data by the business rules.

[0052] It's important to note that the expert rule model described above is a collection of rules (i.e., business rules) built to implement functions such as anomaly detection, operational assistance, and service quality inspection in scenarios like customer service centers. These business rules, based on domain expertise, are designed to retrieve and analyze the distance and location information between specific keywords from massive amounts of long text data to support complex business logic decisions.

[0053] It should be understood that business rules may contain complex logical relationships. For complex business rules, searching long text data directly based on the business rules may result in low retrieval efficiency. Therefore, this embodiment proposes a method of converting business rules into a number of rule elements, each of which represents a rule for searching a business rule in long text data. This is equivalent to simplifying complex business rules so that the subsequent retrieval process can be simplified, thereby improving retrieval efficiency.

[0054] In the specific implementation, you can refer to Figure 2 , Figure 2 This is an example diagram of converting business rules into rule elements in the data retrieval method of this application. Figure 2 In the example diagram of converting business rules into rule elements, the business rule of "((Homepage#Borrow money#Get credit limit#30)|(One month later#Reapply#Credit limit#30))&(Flash loan))|(((Homepage#Borrow money#Get credit limit#30)|(One month later#Reapply#Credit limit#30))&(!(Restore credit limit)|(Credit limit restoration)|(Repayment plan)))" can be converted into the AND, OR, and NOT logical operations of 8 rule elements. These 8 rule elements correspond to Figure 2 In the examples, there are "Homepage#Borrowing#Get#30", "One month later#Reapply#30#Limit", "Flash Loan", "Homepage#Borrowing#Get#30#Limit", "One month later#Reapply#30#Limit", "Restore#Limit", "Limit Restore", "Repayment Plan". Root represents the root node, Node represents the child node, # represents the connector between keywords, and 30 indicates that the interval between keywords cannot exceed 30 characters.

[0055] Step S30: Generate a queryable statement according to the rule element, and search the long text data based on the queryable statement and the inverted index.

[0056] It should be noted that the queryable statements above represent DSL (Domain-Specific Language) statements that can be executed in Elasticsearch. Compared to conventional statements, DSL statements can more concisely and directly express the operational intent of the domain (such as full-text search, filtering, and aggregation). Elasticsearch is an open source, distributed engine built on Lucene that supports real-time search and analysis.

[0057] In a specific implementation, after a queryable statement is determined based on the rule element, it can be executed through Elasticsearch, thereby telling Elasticsearch the precise commands for what to find (e.g., which keywords) and how to find them (e.g., what order, distance, and logical relationship must be satisfied between keywords) using the inverted index (particularly the location information stored in the inverted index). For example, when searching for the keyword "artificial intelligence," this embodiment can directly find a list of document IDs containing this keyword in the long text data through the inverted index, without having to search the entire text of the long text data as traditional methods do, thereby improving search efficiency.

[0058] This embodiment performs word segmentation on long text data, generates an inverted index corresponding to the long text data based on the word segmentation results; converts business rules contained in the expert rule model into rule elements, wherein the rule elements represent rules for searching the long text data by the business rules; generates queryable statements based on the rule elements, and searches the long text data based on the queryable statements and the inverted index. The above method of this embodiment structures the word segmentation results of long text data through an inverted index, and generates queryable statements based on the rule elements converted from the business rules, thereby realizing the abstract expression of complex logic to enhance the manageability and reusability of business rules, thereby enabling accurate retrieval in massive amounts of long text data by combining the inverted index with the queryable statements.

[0059] refer to Figure 3 , Figure 3 This is a flow chart of the second embodiment of the data retrieval method of this application.

[0060] In a feasible implementation, step S10 may include:

[0061] Step S101: performing word segmentation processing on long text data in units of characters, and determining basic information corresponding to a single character in the long text data according to the word segmentation result, the basic information including document ID, offset and number of occurrences.

[0062] Step S102: constructing a triple of the single character according to the basic information, and generating an inverted index corresponding to the long text data according to the triples of all characters in the long text data, wherein the triple represents a positioning constraint of the single character in the long text data.

[0063] It should be noted that the above-mentioned document ID uniquely identifies an indexed long text (such as a customer service conversation record), which can be used to quickly locate documents containing a specified single character; the above-mentioned offset represents the character-level position of a single character in the document, which can be used for precise position and distance constraint retrieval; the above-mentioned number of occurrences represents the total number of times a single character appears in the same document.

[0064] It should be understood that a single character corresponds to a Term (keyword element), and each Term corresponds to an inverted list (PL). This embodiment can store the document ID, offset, and number of occurrences of the Term in all documents in each inverted list, and then combine all inverted lists to construct the above-mentioned inverted index. In addition, in the inverted index, keywords can be located through the TD (Term Dictionary) and constraints can be imposed through the position offset information stored in the PL. This solves the problem of strong three-way retrieval of position, distance, and keywords in massive text, and has high performance and can be used in actual business scenarios.

[0065] In a feasible implementation, step S20 may include:

[0066] Step S201: Decomposing the business rules contained in the expert rule model to obtain domain keywords and target rankings and target distance tolerances corresponding to the domain keywords.

[0067] Step S202: determining a rule element corresponding to the business rule according to the domain keyword, the target ranking, and the target distance tolerance.

[0068] It should be noted that the above-mentioned domain keywords can be customized. For example, in the financial field, "flash loan" can be regarded as a keyword for the entire domain, avoiding the traditional method of splitting "flash loan" into "flash" and "loan", which leads to changes in semantics.

[0069] It should be understood that the target ranking can be sorted based on natural language expressions, and the target distance tolerance can represent the distance range allowed between keyword A and keyword B. For example, for the business rule "One month later #re-#apply #credit#30", the connector "#" can determine that the corresponding target ranking is one month later → re-→apply →credit, and the corresponding target distance tolerance can be a distance range of 0 to 30 characters.

[0070] In a specific implementation, the business rules contained in the expert rule model can be decomposed using a tree node decomposition method. Specifically, the nodes are decomposed using a stack, and pruning optimization is performed based on the consistency of logical connectors at the same level to reduce tree node traversal. The decomposed expert rule model is essentially a multi-branch tree of "rule elements." This embodiment can use a breadth-first algorithm for parsing: starting from the root node, the tree nodes at each level are split into subtrees. After parsing the nodes at the same level, the rules are merged upward.

[0071] Step S203: setting an identification ID for the first rule element, and establishing a mapping relationship between the first parsing result corresponding to the first rule element and the identification ID.

[0072] Step S204: when traversing the business rules included in the expert rule model, if a second rule element identical to the first rule element is found, the first parsing result is used as the parsing result of the second rule element based on the mapping relationship.

[0073] It should be understood that when processing tree nodes, parsing the same "rule element" is redundant. To improve algorithm performance, a unique ID (the aforementioned identification ID) is generated based on the "rule element" during traversal processing, and a dictionary mapping between the ID and the parsing result is recorded as a cache. Before parsing, the result is retrieved from the cache. If the result is not available, the parsing operation is repeated and stored in the dictionary.

[0074] This embodiment performs word segmentation on long text data in units of characters, determines basic information corresponding to a single character in the long text data based on the word segmentation result, and the basic information includes a document ID, an offset, and a number of occurrences; constructs a triple of the single character based on the basic information, and generates an inverted index corresponding to the long text data based on the triples of all characters in the long text data, wherein the triple represents a positioning constraint of the single character in the long text data; decomposes the business rules contained in the expert rule model to obtain domain keywords and target rankings and target distance tolerances corresponding to the domain keywords; determines a rule element corresponding to the business rule based on the domain keywords, the target ranking, and the target distance tolerance; sets an identification ID for a first rule element, and establishes a mapping relationship between a first parsing result corresponding to the first rule element and the identification ID; when traversing the business rules contained in the expert rule model, if a second rule element identical to the first rule element is traversed, the first parsing result is used as the parsing result of the second rule element based on the mapping relationship. The above method in this embodiment adopts character granularity modeling, combined with triple information of document ID, offset and number of occurrences, significantly improves the positioning accuracy of long texts, and provides underlying support for complex retrieval scenarios; at the same time, by deconstructing the business rules contained in the expert rule model into rule elements with sorting constraints and distance tolerance, and establishing a parsing result reuse mechanism, redundant calculation of repeated rules is avoided, thereby improving the parsing speed of complex business logic.

[0075] refer to Figure 4 , Figure 4 This is a flow chart of the third embodiment of the data retrieval method of this application.

[0076] In a feasible implementation, step S30 may include:

[0077] Step S301: Decompose the domain keyword in the rule element into a number of keyword elements, and perform binding constraints on the keyword elements to obtain a basic query unit.

[0078] It should be understood that for searching a single domain keyword, the keyword elements obtained by decomposing it should be continuous to ensure that the meaning of the domain keyword remains unchanged. Therefore, in this embodiment, the keyword elements can be bound and constrained. That is, the span (distance tolerance between keyword elements) can be set to 0 to achieve a span term (indicating an exact match) for any word or sentence constraint.

[0079] Step S302: Generate a queryable statement according to the basic query units corresponding to all the field keywords, the target ranking, and the target distance tolerance, and retrieve the long text data based on the queryable statement and the inverted index.

[0080] In the specific implementation, you can refer to Figure 5 , Figure 5 Schematic diagram of the retrieval process of the data retrieval method of this application. Figure 5 In the text data, the text data may specifically refer to the long text data mentioned in this embodiment, or may be any other text data, which is not limited in this embodiment; the custom Analyze may indicate word segmentation processing of the text data; the Span Term (i.e., corresponding to the above-mentioned basic query unit) indicates a word or phrase that accurately matches a document, and can locate the exact position in the document; and the SpanNear indicates the relative position between multiple Span Terms in a full-text search. It should be noted that the Span Term of all keywords in the rule element can be combined into a Span Near, and slop can be set as the distance tolerance corresponding to the rule element. Finally, this Span Near is used to generate a queryable statement, and the long text data can be retrieved based on the queryable statement and the inverted index.

[0081] In a feasible implementation, the data retrieval method may further include:

[0082] Step S40: When a retrieval task for the long text data is received, the CPU utilization rate and thread pool queue backlog of the currently available instance are obtained.

[0083] Step S50: selecting a target instance from the currently available instances according to the CPU utilization and the thread pool queue backlog, and scheduling the target instance to process the retrieval task.

[0084] In the specific implementation, in order to ensure the smooth operation of the model task under massive data, this embodiment designs a multi-instance adaptive dynamic load scheduling mechanism. Figure 6 , Figure 6This is a schematic diagram of the multi-instance adaptive dynamic load scheduling mechanism of the data retrieval method of this application. The multi-instance adaptive dynamic load scheduling mechanism includes a periodic handshake of dynamic load information: when the identification service is started, each instance will register a unique identifier in the task scheduling center. Subsequently, each instance will synchronize the CPU utilization in the process and the backlog number in the thread pool queue to the task scheduling center through a periodic handshake, and record it in the registry. Task adaptive scheduling compensation: in the allocation link, the task scheduling center sorts and calculates according to the instance status and load information in the registry (i.e., the corresponding CPU utilization and thread pool queue backlog number), screens and allocates it to the instance with carrying capacity (i.e., the target instance). During the task execution process, the task scheduling center monitors the operation status of each instance and the status of the distributed tasks in real time, and decides on the execution or re-run of the task through the built-in state machine. If the instance has died, the unexecuted tasks will be reallocated. Distributed current limiter based on priority queue: This embodiment uses priority queue + distributed current limiter to ensure that tasks run smoothly according to priority. The queue leverages the binary heap data structure to calculate priority based on the number of compensations, then performs up-heap and down-heap operations to maintain task priority. The distributed rate limiter is based on the token bucket algorithm. A key is set in Redis, and the key value is a counter. A certain number of tokens are periodically added to the counter. When a request arrives, one token is subtracted. If the number of tokens is less than 0, the request is rejected. Task represents the retrieval task, and Priority represents the priority.

[0085] This embodiment decomposes the domain keywords in the rule element into several keyword elements, and binds and constrains the keyword elements to obtain a basic query unit; generates a queryable statement based on the basic query units corresponding to all the domain keywords, the target ranking and the target distance tolerance, and retrieves the long text data based on the queryable statement and the inverted index; when receiving a retrieval task for the long text data, obtains the CPU utilization and thread pool queue backlog of the current available instance; selects a target instance from the current available instance based on the CPU utilization and the thread pool queue backlog, and schedules the target instance to process the retrieval task. The above method of this embodiment is based on the keyword element decomposition and binding constraint mechanism, converts complex domain queries into structured basic units, combines target ranking and distance tolerance to generate precise query statements, and cooperates with inverted index technology to achieve a dual improvement in retrieval speed and accuracy; at the same time, by real-time monitoring of CPU utilization and thread pool backlog status, intelligently matches the optimal processing instance, effectively avoids resource contention and task accumulation, and ensures retrieval response time in high concurrency scenarios.

[0086] Reference Figure 7 , Figure 7This is a structural block diagram of the first embodiment of the data retrieval device of this application.

[0087] like Figure 7 As shown, the data retrieval device proposed in the embodiment of the present application includes:

[0088] An index generation module 701 is configured to perform word segmentation on the long text data and generate an inverted index corresponding to the long text data based on the word segmentation result;

[0089] A rule parsing module 702 is used to convert the business rules contained in the expert rule model into rule elements, wherein the rule elements represent the rules for searching the long text data by the business rules;

[0090] The data retrieval module 703 is configured to generate a queryable statement according to the rule element, and search the long text data based on the queryable statement and the inverted index.

[0091] This embodiment performs word segmentation on long text data, generates an inverted index corresponding to the long text data based on the word segmentation results; converts business rules contained in the expert rule model into rule elements, wherein the rule elements represent rules for searching the long text data by the business rules; generates queryable statements based on the rule elements, and searches the long text data based on the queryable statements and the inverted index. The above method of this embodiment structures the word segmentation results of long text data through an inverted index, and generates queryable statements based on the rule elements converted from the business rules, thereby realizing the abstract expression of complex logic to enhance the manageability and reusability of business rules, thereby enabling accurate retrieval in massive amounts of long text data by combining the inverted index with the queryable statements.

[0092] Based on the first embodiment of the data retrieval device of the present application, a second embodiment of the data retrieval device of the present application is proposed.

[0093] In this embodiment, the index generation module 701 is also used to perform word segmentation processing on long text data in units of characters, and determine the basic information corresponding to a single character in the long text data based on the word segmentation result, and the basic information includes document ID, offset and number of occurrences; construct a triple of the single character based on the basic information, and generate an inverted index corresponding to the long text data based on the triples of all characters in the long text data, and the triple represents the positioning constraint of the single character in the long text data.

[0094] Furthermore, the rule parsing module 702 is also used to disassemble the business rules contained in the expert rule model to obtain domain keywords and the target ranking and target distance tolerance corresponding to the domain keywords; and determine the rule element corresponding to the business rule based on the domain keywords, the target ranking and the target distance tolerance.

[0095] Furthermore, the rule parsing module 702 is also used to set an identification ID for the first rule element and establish a mapping relationship between the first parsing result corresponding to the first rule element and the identification ID; when traversing the business rules contained in the expert rule model, if a second rule element that is the same as the first rule element is traversed, the first parsing result is used as the parsing result of the second rule element based on the mapping relationship.

[0096] Furthermore, the data retrieval module 703 is also used to decompose the domain keywords in the rule element into several keyword elements, and bind the keyword elements to obtain basic query units; generate a queryable statement based on the basic query units corresponding to all the domain keywords, the target ranking and the target distance tolerance, and retrieve the long text data based on the queryable statement and the inverted index.

[0097] Furthermore, the data retrieval module 703 is also used to obtain the CPU utilization and thread pool queue backlog of the currently available instance when receiving a retrieval task for the long text data; select a target instance from the currently available instances based on the CPU utilization and the thread pool queue backlog, and schedule the target instance to process the retrieval task.

[0098] Other embodiments or specific implementations of the data retrieval device of the present application can refer to the above-mentioned method embodiments and will not be described in detail here.

[0099] The present application provides a data retrieval device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the data retrieval method in the above-mentioned embodiment 1.

[0100] Reference below Figure 8, which shows a schematic diagram of the structure of a data retrieval device suitable for implementing the embodiments of the present application. The data retrieval device in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 8 The data retrieval device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0101] like Figure 8 As shown, the data retrieval device may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes based on programs stored in a read-only memory 1002 or programs loaded from a storage device 1003 into a random access memory 1004. Random access memory 1004 also stores various programs and data required for the operation of the data retrieval device. The processing device 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: an input device 1007 including, for example, a touch screen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, hard disk, etc.; and a communication device 1009. Communication device 1009 can allow data retrieval equipment to carry out wireless or wired communication with other equipment to exchange data.Although the data retrieval equipment with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown.Can implement or have more or less systems alternatively.

[0102] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 1003, or installed from a read-only memory 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are performed.

[0103] The data retrieval device provided in this application, utilizing the data retrieval method of the aforementioned embodiment, can resolve the technical problem of the prior art in being unable to accurately retrieve data from massive amounts of long text. Compared to the prior art, the beneficial effects of the data retrieval device provided in this application are the same as those of the data retrieval method provided in the aforementioned embodiment. Other technical features of this data retrieval device are the same as those disclosed in the aforementioned embodiment and are not further elaborated here.

[0104] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0105] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0106] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer program) stored thereon, wherein the computer-readable program instructions are used to execute the data retrieval method in the above embodiment.

[0107] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0108] The computer-readable storage medium may be included in the data retrieval device, or may exist independently without being incorporated into the data retrieval device.

[0109] The computer-readable storage medium carries one or more programs, which, when executed by the data retrieval device, enable the data retrieval device to write computer program code for performing the operations of the present application in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++; and also conventional procedural programming languages ​​such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, such as a local area network (LAN) or a wide area network (WAN), or connected to an external computer (for example, through the Internet using an Internet service provider).

[0110] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0111] The modules described in the embodiments of the present application may be implemented in software or hardware, wherein the name of a module does not necessarily limit the unit itself.

[0112] The computer-readable storage medium provided in this application stores computer-readable program instructions (i.e., a computer program) for executing the aforementioned data retrieval method. This computer-readable storage medium can address the technical issue of prior art inability to accurately retrieve data from massive amounts of long text data. Compared to prior art, the beneficial effects of the computer-readable storage medium provided in this application are similar to those of the data retrieval method provided in the aforementioned embodiments and are not further elaborated here.

[0113] The present application also provides a computer program product, comprising a computer program, which implements the steps of the above-mentioned data retrieval method when executed by a processor.

[0114] The computer program product provided in this application can solve the technical problem of data retrieval. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as the beneficial effects of the data retrieval method provided in the above embodiment, which will not be repeated here.

[0115] The above description is only part of the embodiments of the present application and does not limit the patent scope of the present application. All equivalent structural transformations made by using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A data retrieval method, characterized in that: The method comprises the following steps: Perform word segmentation on the long text data, and generate an inverted index corresponding to the long text data based on the word segmentation result; Converting the business rules included in the expert rule model into rule elements, wherein the rule elements represent rules for performing retrieval of the business rules in the long text data; A queryable statement is generated according to the rule element, and the long text data is retrieved based on the queryable statement and the inverted index.

2. The data retrieval method according to claim 1, wherein: The step of performing word segmentation on the long text data and generating an inverted index corresponding to the long text data according to the word segmentation result includes: Perform word segmentation on the long text data in character units, and determine basic information corresponding to a single character in the long text data based on the word segmentation result, the basic information including document ID, offset, and number of occurrences; A triplet of the single character is constructed according to the basic information, and an inverted index corresponding to the long text data is generated according to the triplet of all characters in the long text data, wherein the triplet represents a positioning constraint of the single character in the long text data.

3. The data retrieval method according to claim 1, wherein: The step of converting the business rules contained in the expert rule model into rule elements includes: Decomposing the business rules contained in the expert rule model to obtain domain keywords and target rankings and target distance tolerances corresponding to the domain keywords; A rule element corresponding to the business rule is determined according to the domain keyword, the target ranking, and the target distance tolerance.

4. The data retrieval method according to claim 3, wherein: After the step of determining the rule element corresponding to the business rule according to the domain keyword, the target ranking and the target distance tolerance, the method further includes: Setting an identification ID for the first rule element, and establishing a mapping relationship between the first parsing result corresponding to the first rule element and the identification ID; When traversing the business rules included in the expert rule model, if a second rule element that is the same as the first rule element is traversed, the first parsing result is used as the parsing result of the second rule element based on the mapping relationship.

5. The data retrieval method according to claim 3, wherein: The step of generating a queryable statement according to the rule element and searching the long text data based on the queryable statement and the inverted index includes: Decomposing the domain keyword in the rule element into a plurality of keyword elements, and binding constraints on the keyword elements to obtain a basic query unit; A queryable statement is generated according to the basic query units corresponding to all the field keywords, the target ranking, and the target distance tolerance, and the long text data is retrieved based on the queryable statement and the inverted index.

6. The data retrieval method according to claim 1, wherein: The method further comprises: When receiving a retrieval task for the long text data, obtaining the CPU utilization rate and thread pool queue backlog of the currently available instance; A target instance is selected from the currently available instances according to the CPU utilization and the thread pool queue backlog, and the target instance is scheduled to process the retrieval task.

7. A data retrieval device, characterized in that: The data retrieval device comprises: An index generation module is used to perform word segmentation on long text data and generate an inverted index corresponding to the long text data based on the word segmentation results; A rule parsing module, configured to convert the business rules contained in the expert rule model into rule elements, wherein the rule elements represent rules for performing retrieval of the business rules in the long text data; A data retrieval module is used to generate a queryable statement according to the rule element, and retrieve the long text data based on the queryable statement and the inverted index.

8. A data retrieval device, characterized in that: The device includes: a memory, a processor, and a data retrieval program stored in the memory and executable on the processor, wherein the data retrieval program is configured to implement the steps of the data retrieval method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a data retrieval program is stored on the storage medium. When the data retrieval program is executed by a processor, the steps of the data retrieval method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product includes a data retrieval program, and when the data retrieval program is executed by a processor, the steps of the data retrieval method according to any one of claims 1 to 6 are implemented.

Images