Query processing method and device, equipment, storage medium and program product

Abstract

The invention provides a query processing method which can be applied to the technical field of artificial intelligence. The query processing method comprises the steps that an original query text is acquired and preprocessed; inputting the preprocessed original query text into a pre-trained target model to obtain an entity of the original query text; performing standard normalization on the entity of the original query text; performing query expansion and reconstruction on the entity after standard normalization to obtain executable query representation; based on a target query template determined by the intention of the executable query representation, filling placeholders of the target query template with the executable query representation to obtain a target query statement for retrieval; wherein the target model is a double-architecture model containing a generation path and a compatible path. The invention further provides a query processing device and equipment, a storage medium and a program product.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and more particularly to a query processing method and device, equipment, a storage medium and a program product. BACKGROUND

[0002] In the current financial field, with the rapid development of information technology, the quantity and complexity of financial data have increased dramatically.

[0003] When a user performs a financial query, the user often inputs a complex query request using natural language. Existing financial query processing techniques often rely on rule matching or keyword-based retrieval, which is low in accuracy and efficiency when processing complex queries. In addition, the dynamic changes of financial data make it difficult for static query templates to meet the real-time needs of users, resulting in poor user experience. SUMMARY

[0004] In view of the above problems, the present application provides a query processing method, device, equipment, storage medium and program product.

[0005] According to a first aspect of the present application, a query processing method is provided, comprising: obtaining and preprocessing an original query text; inputting the preprocessed original query text into a pre-trained target model to obtain entities of the original query text; standardizing the entities of the original query text; performing query expansion and reconstruction on the standardized entities to obtain an executable query representation; filling the executable query representation into a placeholder of a target query template determined based on an intent of the executable query representation to obtain a target query sentence for retrieval; wherein the target model is a dual architecture model containing a generation path and a compatible path.

[0006] According to an embodiment of the present application, obtaining and preprocessing the original query text comprises: cleaning and normalizing the original query text; determining a target candidate word in a preset candidate word set based on a language model of a pseudo-log-likelihood probability; and performing a misspelling correction on the normalized original query text based on the target candidate word.

[0007] According to an embodiment of the present application, before inputting the preprocessed original query text into the pre-trained target model, it further comprises: constructing the original query text into a plurality of text segments; and after inserting an identifier in front of each text segment, performing a word segmentation on the text segments based on a preset sliding window.

[0008] According to an embodiment of this application, the target model is a dual-architecture model comprising a generative model and a discriminative model, wherein the generative model executes a generative path and the discriminative model executes a compatibility path; and the generative model includes a sequence-to-sequence generator, and the discriminative model is a discriminative model based on a bidirectional encoder and a conditional random field.

[0009] According to an embodiment of this application, the target model is obtained through pre-training, the sequence-to-sequence generator includes an encoder and a decoder, and the training process includes: the encoder receiving multiple text segments to be trained and generating context vectors; based on a constraint generation and copying mechanism, the decoder decodes the context vectors to obtain the entities of the text to be trained.

[0010] According to an embodiment of this application, the target model is pre-trained, and the training process further includes: when the confidence of the generation path is insufficient, training the text segment to be trained based on the discriminant of the bidirectional encoder and the conditional random field to obtain the entity of the text to be trained.

[0011] According to an embodiment of this application, the entities of the original query text are standardized, including: determining the type of the entity based on the entity's label; and performing standard normalization processing based on the entity type, where the entity types include time entities, name entities, and amount entities.

[0012] According to embodiments of this application, query expansion and reconstruction are performed on the standardized entities to obtain an executable query representation, including: establishing a graph candidate set; judging the graph candidate set based on a preset expansion discriminator to determine the graph candidate set that can be used for expansion; and performing query expansion and reconstruction on the standardized entities based on the graph candidates that can be used for expansion to obtain an executable query representation.

[0013] A second aspect of this application provides a query processing apparatus, comprising: a raw query text acquisition module for acquiring and preprocessing raw query text; an entity acquisition module for inputting the preprocessed raw query text into a pre-trained target model to obtain entities of the raw query text; a standard normalization processing module for standard normalizing the entities of the raw query text; an executable query representation acquisition module for query expansion and reconstruction of the standard normalized entities to obtain an executable query representation; and a target query statement acquisition module for filling the executable query representation into placeholders in the target query template based on the intent of the executable query representation to obtain a target query statement for retrieval; wherein the target model is a dual-architecture model containing a generation path and a compatibility path.

[0014] A third aspect of this application provides an electronic device comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method described above.

[0015] A fourth aspect of this application also provides a computer-readable storage medium having a computer program or instructions stored thereon, which, when executed by a processor, implement the steps of the above-described method.

[0016] The fifth aspect of this application also provides a computer program product, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described method. Attached Figure Description

[0017] The above-mentioned contents, other objects, features and advantages of this application will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0018] Figure 1 This illustration schematically depicts an application scenario of the query processing method according to an embodiment of this application;

[0019] Figure 2 A flowchart illustrating a query processing method according to an embodiment of this application is shown schematically.

[0020] Figure 3 This illustration schematically shows a flowchart of preprocessing the original query text according to an embodiment of this application;

[0021] Figure 4 This illustration shows a flowchart of training a target model according to an embodiment of the present application;

[0022] Figure 5 The flowchart illustrating query expansion and reconstruction of entities according to embodiments of this application is shown in the illustration.

[0023] Figure 6 A schematic diagram illustrating the structure of a query processing apparatus according to an embodiment of this application is shown; and

[0024] Figure 7 A block diagram schematically illustrates an electronic device suitable for implementing a query processing method according to an embodiment of this application. Detailed Implementation

[0025] The embodiments of this application will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this application. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of this application for ease of explanation. However, it will be apparent that one or more embodiments may be implemented without these specific details. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0026] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The terms “comprising,” “including,” etc., as used herein indicate the presence of features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0027] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0028] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0029] In the technical solution of this application, the user information (including but not limited to user personal information, user image information, user device information, such as location information) and data (including but not limited to data used for analysis, stored data, and displayed data) involved are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of related data all comply with relevant laws, regulations, and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entry points for users to choose to authorize or refuse.

[0030] In scenarios involving automated decision-making using personal information, the methods, devices, and systems provided in this application all offer users corresponding entry points for choosing to agree to or reject the automated decision-making results. If the user chooses to reject, the process proceeds to the expert decision-making stage. Here, "automated decision-making" refers to the activity of automatically analyzing and evaluating an individual's behavioral habits, interests, or economic, health, and credit status through computer programs, and then making a decision. Here, "expert decision-making" refers to the activity of making decisions by personnel who specialize in a particular field, possess specialized experience, knowledge, and skills, and have reached a certain level of professional expertise.

[0031] Embodiments of this application provide a query processing method, including: acquiring and preprocessing the original query text; inputting the preprocessed original query text into a pre-trained target model to obtain entities of the original query text; standardizing the entities of the original query text; performing query expansion and reconstruction on the standardized entities to obtain an executable query representation; filling the executable query representation into placeholders in the target query template based on the intent of the executable query representation to obtain a target query statement for retrieval; wherein, the target model is a dual-architecture model containing a generation path and a compatibility path.

[0032] Through the embodiments of this application, the method employs a dual-architecture model (a combination of a generative model and a discriminative model) to better capture the intent of the query text, thereby improving the accuracy and efficiency of the query. Furthermore, by utilizing query expansion and reconstruction techniques, the user's initial query can be transformed into richer and more diverse query formats, thus improving the coverage and effectiveness of the retrieval. This method not only efficiently processes query requests but also addresses complex query needs with multiple semantic layers.

[0033] Figure 1 The diagram illustrates an application scenario of the query processing method according to an embodiment of this application.

[0034] like Figure 1 As shown, application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, and a server 105. Network 104 serves as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. Network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0035] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0036] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0037] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0038] It should be noted that the query processing method provided in this application embodiment can generally be executed by server 105. Correspondingly, the query processing device provided in this application embodiment can generally be located in server 105. The query processing method provided in this application embodiment can also be executed by a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105. Correspondingly, the query processing device provided in this application embodiment can also be located in a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105.

[0039] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0040] The following will be based on Figure 1 The described scene, through Figures 2-5 The query processing method according to the embodiments of this application will be described in detail.

[0041] Figure 2 A flowchart illustrating a query processing method according to an embodiment of this application is shown.

[0042] likeFigure 2 As shown, according to an embodiment of this application, the query processing method specifically includes operations S210 to S250.

[0043] In operation S210, the original query text is obtained and preprocessed.

[0044] In the embodiments of this application, after obtaining the original query text, a series of text standardization operations are performed for preprocessing. These operations may include, for example, character set unification, case conversion, language variant unification, and error token correction, to transform the diverse query text input by the user into a unified, computable internal representation. The preprocessing process is described in detail below.

[0045] Figure 3 The flowchart illustrating the preprocessing of the original query text according to an embodiment of this application is shown schematically.

[0046] like Figure 3 As shown, according to an embodiment of this application, the original query text is obtained and preprocessed, which specifically includes operations S310 to S330.

[0047] In operation S310, the original query text is cleaned and standardized.

[0048] For example, the system first receives the original query text submitted by the user, which contains various non-standard characters, mixed case, traditional Chinese characters, or typos and other noise information.

[0049] For example, the received raw text can be converted from full-width characters to half-width characters to eliminate interference from full-width characters caused by differences in input methods or character encoding. Therefore, standardizing to half-width characters ensures standardized character representation, which is beneficial for subsequent character- or word-based matching and analysis. Furthermore, after the full-width to half-width conversion, all English letters in the text can be uniformly converted between uppercase and lowercase.

[0050] For example, considering the diverse input habits of Chinese users, queries containing Traditional Chinese characters will be uniformly converted to Simplified Chinese using pre-defined Simplified-Traditional conversion rules or by calling a professional conversion library. This operation ensures that all Chinese queries are processed internally in Simplified Chinese, avoiding semantic misunderstandings caused by the mixing of Simplified and Traditional characters.

[0051] In operation S320, a language model based on pseudo-log-likelihood probability is used to determine target candidate words in a preset candidate word set.

[0052] In operation S330, based on the target candidate words, the normalized original query text is corrected for typos.

[0053] In the embodiments of this application, after character normalization and simplified / traditional conversion are completed, important typo correction processing is performed. In this embodiment, a two-stage typo correction process is adopted, combining candidate word generation based on edit distance and candidate word selection based on language model probability. In particular, optimization is performed on vocabulary specific to the financial field, business terms, and common spelling errors to ensure the semantic accuracy of the input query.

[0054] Specifically, based on a pre-set high-confidence dictionary, target words whose edit distance to each word to be corrected in the input original query text is less than or equal to a pre-set threshold are searched, and a candidate word set is established based on the target words that meet the conditions.

[0055] Furthermore, a language model based on pseudo-log-likelihood probability is used to determine the target candidate word in the candidate word set. It can be understood that the target candidate word here is the optimal candidate word.

[0056] Furthermore, after selecting target candidate words, the words to be corrected are replaced with target candidate words, ultimately outputting a clean and standardized query text. This query text does not contain noise from the original query, and the characters and language are consistent, with typos corrected. Therefore, it can be directly used as input for subsequent deep learning models to improve the accuracy and efficiency of model processing.

[0057] Through the embodiments of this application, by preprocessing the original query text, it is ensured that the user's intent can be accurately understood, reducing the possibility of misunderstanding and erroneous responses.

[0058] In operation S220, the preprocessed original query text is input into the pre-trained target model to obtain the entity of the original query text.

[0059] In the embodiments of this application, the preprocessed original query text is input into the target model, and the target model performs deep semantic parsing on the text to extract structured entities, so as to accurately convert it into operable structured query text.

[0060] In the embodiments of this application, the target model is a dual-architecture deep learning model that includes a generative path and a compatibility path.

[0061] Specifically, the target model is a dual-architecture model that includes a generative model and a discriminative model. The generative model executes the generative path, and the discriminative model executes the compatibility path. The generative model includes a sequence-to-sequence generator, and the discriminative model is a discriminative model based on a bidirectional encoder and a conditional random field.

[0062] Understandably, in order to achieve a balance between high accuracy and rapid adaptation to data changes, the target model adopts a dual architecture design with a generative model as the main component and a discriminative model as a backup. It ensures output accuracy through constrained generation and replication mechanisms, enhances semantic understanding by utilizing multi-source context fusion, and achieves rapid mapping of new fields by leveraging vectorized zero-shot matching and online adaptation.

[0063] In the embodiments of this application, the target model is pre-trained, and the training process of the target model is described in detail below.

[0064] It should be noted that before inputting the preprocessed original query text into the pre-trained target model, the process also includes: constructing the original query text into multiple text segments; inserting an identifier in front of each text segment, and then performing word segmentation on the text segment based on a preset sliding window.

[0065] Specifically, to ensure a consistent input format for the target model, the original query text is constructed into different text segments, each preceded by a specific identifier to help the encoder distinguish the source of information. At the encoding level, segment embedding and round position embedding are used to differentiate between source and order.

[0066] Furthermore, for short texts, a vocabulary size of 32k and a maximum sequence length of 512 are used for word segmentation. For very long texts, a sliding window strategy is used for word segmentation to ensure that the current query and recent context are always preserved, reducing misunderstandings and erroneous responses.

[0067] Figure 4 The flowchart illustrating the training of the target model according to an embodiment of this application is shown schematically.

[0068] like Figure 4 As shown, according to an embodiment of this application, the target model is obtained through pre-training, and the sequence-to-sequence generator includes an encoder and a decoder. This training process specifically includes operations S410 to S420.

[0069] In operation S410, the encoder receives multiple text segments to be trained and generates context vectors.

[0070] In operation S420, based on the constraint generation and copying mechanism, the decoder decodes the context vector to obtain the entities of the text to be trained.

[0071] In the embodiments of this application, the generative model in the target model first executes a generative path, which includes a sequence-to-sequence generator, which is an encoder-decoder structure.

[0072] Specifically, the encoder receives multiple text segments to be trained and generates context vectors, which the decoder dynamically references during generation through cross-attention. To ensure that entities are accurately parsed from the history or retrieved segments and precisely aligned with the original text, the decoder integrates a copy / pointer mechanism.

[0073] Furthermore, to prevent format errors in the entity results generated by the decoder, a constrained decoder is employed. Based on the bundle search, a finite state machine is used to verify each path; if a path triggers a violation, it is immediately pruned. Additionally, the average log probability of each generated context vector is recorded, and the sequence confidence is calculated accordingly. If the final confidence score falls below a set threshold, a fallback to a compatible path or post-processing clarification is triggered. Unlike traditional discriminative methods, this generation path directly outputs a structured list of entities (containing intents and slots) that can be used for subsequent operations and returns a proof field to ensure result traceability.

[0074] In the embodiments of this application, when the confidence of the generated path is insufficient, the text segment to be trained is trained based on the discriminant of the bidirectional encoder and the conditional random field to obtain the entity of the text to be trained.

[0075] Specifically, when the confidence level of the generated path is insufficient, it falls back to a compatible path, which is a discriminative implementation based on a bidirectional encoder and a conditional random field. This path uses a bidirectional encoding model as the encoder. The vector output by the bidirectional encoding model is passed through a fully connected network and a function layer to output the intent label and confidence level. The encoded context vector embedding is then mapped to the slot label dimension through another fully connected network and input to the sequence labeling layer (i.e., the conditional random field) for joint sequence decoding to obtain the entities of the text to be trained. The optimal label sequence is then solved using a dynamic programming algorithm.

[0076] In the embodiments of this application, in order to achieve the mapping of entities to fields without retraining on each new dataset, a dual mechanism of "vectorized pattern matching + semantic rule enhancement" is also adopted. The entity text extracted by the generation path or the discrimination path is mapped into entity vectors after standardization (synonym reduction, word form restoration, etc.).

[0077] Through the embodiments of this application, the dual-architecture model design can better capture the intent of query text, improving query accuracy and efficiency. It also enables the system to quickly adapt to changing financial data, generating query statements that meet user needs in real time, thus enhancing the user experience.

[0078] In operation S230, the entities in the original query text are standardized.

[0079] In the embodiments of this application, the entities of the original query text are standardized, including: determining the type of the entity based on the entity's label; and performing standard normalization processing based on the entity type, where the entity types include time entities, name entities, and amount entities.

[0080] Specifically, based on the type label of each extracted entity, it is standardized or normalized accordingly. A rule-based routing mechanism ensures that different types of entities can be processed by their respective dedicated logic.

[0081] For example, when performing standard normalization on time entity types, colloquial or vague time expressions are converted into precise, machine-understandable time points or time periods. Natural language contains many vague and relative time expressions, such as "last month," "last week," and "next Tuesday," the precise meaning of which depends on the current context. Therefore, this embodiment constructs a complex and robust time expression parser. This parser is based on a set of context-free grammar rules and uses the current system time as an anchor for dynamic computation. The rule set defines the syntactic structure of various time phrases and associates them with corresponding semantic computation logic. For expressions like "Wednesday," the parser disambiguates based on the tense in the query to determine whether it refers to the previous Wednesday or the upcoming Wednesday.

[0082] For example, when standardizing entity types such as person names and organization names, the ambiguity of proper nouns like these is eliminated, and they are linked to unique, structured entity identifiers in the backend knowledge base. In natural language, for example, "Zhang San" might refer to multiple people with the same name or historical trading partners in a user's address book. Simple text matching cannot meet business needs, so entity linking is required to eliminate ambiguity. For example, this embodiment can be divided into two steps: First, candidate generation involves searching the user's personal financial knowledge graph (including but not limited to the user's frequently used contact list, historical trading partner information, and linked family accounts) for all candidate entities that match the extracted entity names. Second, disambiguation occurs when multiple candidate entities with the same name exist. The system attempts to disambiguate using other contextual information in the query. For example, if the query includes "mortgage account," entities with the same name that have a "mortgage co-borrower" relationship with the user are prioritized.

[0083] For example, when performing standardization on monetary entity types, the colloquial expressions of monetary amounts in the user-input query text are converted into structured data with standard numerical values ​​and currency units. Users often use colloquial, non-standard, or abbreviated forms when expressing amounts; therefore, this embodiment employs a dual-method standardization process. First, a comprehensive mapping rule and parsing algorithm are established, including the recognition and conversion of uppercase digits, Chinese numerical units, and combined digits. Second, a built-in unit and currency mapping table can recognize various colloquial monetary units and map them to standard currency symbols.

[0084] Furthermore, based on the entity standardization processes described above, a set containing all standardized and normalized entity information is obtained. These structured entities will be compared with the previously identified query intent. Figure 1 This serves as the precise input for subsequent query expansion.

[0085] Through the embodiments of this application, for complex query needs unique to the financial field, such as the combination of multiple entities such as time, amount and name, it is possible to flexibly perform standardized processing and query reconstruction to support diversified queries.

[0086] In operation S240, the standardized entities are expanded and reconstructed to obtain an executable query representation.

[0087] Figure 5 The flowchart illustrating query expansion and reconstruction of entities according to embodiments of this application is shown in the illustration.

[0088] like Figure 5 As shown, according to an embodiment of this application, the standardized entity is extended and reconstructed to obtain an executable query representation. This process specifically includes operations S510 to S530.

[0089] When operating the S510, a candidate set of maps is established.

[0090] In operation S520, the candidate set of the map is judged based on the preset expansion discriminator in order to determine the candidate set of the map that can be used for expansion.

[0091] In operation S530, based on the graph candidates that can be used for expansion, the standardized entities are query expanded and reconstructed to obtain an executable query representation.

[0092] In embodiments of this application, for example, a candidate set (denoted as "graph candidate") is generated by traversing edge types such as "synonym," "hypernym," and "related" in the graph, with each edge assigned a relation weight according to its edge type. Alternatively, the highest cosine similarity item can be retrieved from the pre-trained word / sentence vector space for the query core word as a representation candidate to establish the candidate set. Another example is that the outputs of the two channels can be merged into a candidate set, and semantic compatibility (i.e., the cosine similarity between the candidate and the query core in the vector space) can be calculated for each candidate.

[0093] Furthermore, to avoid blindly including low-quality or falsely invoked extensions, a pre-defined extension discriminator is used to determine whether the candidate set is a suitable extension in the current query context. The extension discriminator adopts a dual-tower encoding structure, where the query encoder and candidate encoder encode the query context and candidate options into vectors, respectively. The discriminator is trained through contrastive learning to output the probability of extension correctness.

[0094] Furthermore, for each graph candidate that has passed the discriminator screening, a comprehensive score calculation is introduced to determine its logical operator and priority, thereby completing the query expansion and reconstruction of the standardized entity to obtain an executable query representation.

[0095] In operation S250, based on the target query template determined by the intent of the executable query representation, the executable query representation is filled into the placeholders of the target query template to obtain the target query statement used for retrieval.

[0096] Specifically, upon receiving an executable query representation, a predefined target query template is selected based on its explicit query intent. These target query templates are one or more highly parameterized templates predefined according to each core business intent. These templates are skeletal structures recognizable by the backend search engine, containing placeholders for subsequent filling in of specific values. For example, if the intent is a transaction query, the system will select a target query template related to transaction queries; if the intent is a product search, it will select a template related to product searches. This template-based approach ensures that queries with different intents generate the correct structure that conforms to their respective business logic.

[0097] Furthermore, after determining a suitable target query template, a dynamic population operation is performed. The standardized entity values ​​and expanded terminology output above precisely replace the placeholders in the template. After placeholder population, a complete target query statement that conforms to the syntax specifications of the target backend search engine is generated. This statement already includes all necessary filtering conditions, matching terms, and possible sorting rules, ensuring the completeness and accuracy of the query.

[0098] Understandably, the generated target query will be sent to the backend search engine or the corresponding database interface. The search engine will execute this query and return data records that meet the criteria. These records will then be passed to subsequent result processing modules for parsing, aggregation, and presentation to the user.

[0099] Through the embodiments of this application, query expansion and reconstruction techniques can be used to transform a user's initial query into a richer and more diverse query format, thereby improving the coverage and effectiveness of the retrieval.

[0100] Based on the above query processing method, this application also provides a query processing apparatus. The following will be combined with... Figure 6 The device is described in detail.

[0101] Figure 6 A schematic block diagram of a query processing apparatus according to an embodiment of this application is shown.

[0102] like Figure 6 As shown, the query processing device 600 of this embodiment includes an original query text acquisition module 610, an entity acquisition module 620, a standard normalization processing module 630, an executable query representation acquisition module 640, and a target query statement acquisition module 650.

[0103] The original query text acquisition module 610 is used to acquire the original query text and perform preprocessing. In one embodiment, the original query text acquisition module 610 can be used to perform the operation S210 described above, which will not be repeated here.

[0104] The entity acquisition module 620 is used to input the preprocessed original query text into a pre-trained target model to obtain the entities of the original query text. In one embodiment, the entity acquisition module 620 can be used to perform the operation S220 described above, which will not be repeated here.

[0105] The standard normalization processing module 630 is used to perform standard normalization on the entities of the original query text. In one embodiment, the standard normalization processing module 630 can be used to perform the operation S230 described above, which will not be repeated here.

[0106] The executable query representation acquisition module 640 is used to perform query expansion and reconstruction on the standardized entities to obtain an executable query representation. In one embodiment, the executable query representation acquisition module 640 can be used to perform the operation S240 described above, which will not be repeated here.

[0107] The target query statement acquisition module 650 is used to fill the placeholders in the target query template with the executable query representation based on the intent of the executable query representation, thereby obtaining the target query statement for retrieval. In one embodiment, the target query statement acquisition module 650 can be used to perform the operation S250 described above, which will not be repeated here.

[0108] According to an embodiment of this application, the target model is a dual-architecture model that includes a generation path and a compatibility path.

[0109] According to an embodiment of this application, the original query text acquisition module 610 includes a standardization processing submodule, a target candidate word determination submodule, and a correction processing submodule.

[0110] The standardization processing submodule is used to clean and standardize the original query text. In one embodiment, the standardization processing submodule can be used to perform the operation S310 described above, which will not be repeated here.

[0111] The target candidate word determination submodule is used to determine target candidate words in a preset candidate word set based on a language model with pseudo-log-likelihood probability. In one embodiment, the target candidate word determination submodule can be used to perform the operation S320 described above, which will not be repeated here.

[0112] The correction processing submodule is used to correct typos in the normalized original query text based on the target candidate words. In one embodiment, the correction processing submodule can be used to perform the operation S330 described above, which will not be repeated here.

[0113] According to an embodiment of this application, the entity acquisition module 620 includes a word segmentation processing submodule, a first target model training submodule, and a second target model training submodule.

[0114] The word segmentation submodule is used to construct multiple text segments from the original query text; after inserting an identifier before each text segment, word segmentation is performed on the text segment based on a preset sliding window.

[0115] The first submodule for training the target model is used by the encoder to receive multiple text segments to be trained and generate context vectors; based on the constraint generation and copying mechanism, the decoder decodes the context vectors to obtain the entities of the text to be trained. In one embodiment, the first submodule for training the target model can be used to perform the operations S410 to S420 described above, which will not be repeated here.

[0116] The second sub-module for training the target model is used to train the text segment to be trained based on the discriminant of the bidirectional encoder and conditional random field when the confidence of the generated path is insufficient, so as to obtain the entity of the text to be trained.

[0117] According to an embodiment of this application, the executable query representation acquisition module 640 includes a graph candidate set establishment submodule, an extended discrimination submodule, and a query extension submodule.

[0118] The map candidate set establishment submodule is used to establish a map candidate set. In one embodiment, the map candidate set establishment submodule can be used to perform the operation S510 described above, which will not be repeated here.

[0119] The extended discrimination submodule is used to discriminate the candidate set of the atlas based on a preset extended discriminator, so as to determine the candidate set of the atlas that can be used for expansion. In one embodiment, the extended discrimination submodule can be used to perform the operation S520 described above, which will not be repeated here.

[0120] The query expansion submodule is used to expand and reconstruct the standardized entities based on the graph candidates that can be used for expansion, thereby obtaining an executable query representation. In one embodiment, the query expansion submodule can be used to perform the operation S530 described above, which will not be repeated here.

[0121] According to embodiments of this application, any multiple modules among the original query text acquisition module 610, entity acquisition module 620, standard normalization processing module 630, executable query representation acquisition module 640, and target query statement acquisition module 650 can be merged into one module, or any one of these modules can be split into multiple modules. Alternatively, at least some of the functions of one or more of these modules can be combined with at least some of the functions of other modules and implemented in one module. According to embodiments of this application, at least one of the original query text acquisition module 610, entity acquisition module 620, standard normalization processing module 630, executable query representation acquisition module 640, and target query statement acquisition module 650 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), programmable logic array (PLA), system-on-a-chip, system-on-a-substrate, system-on-package, application-specific integrated circuit (ASIC), or any other reasonable means of integrating or packaging circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, at least one of the original query text acquisition module 610, entity acquisition module 620, standard normalization processing module 630, executable query representation acquisition module 640, and target query statement acquisition module 650 can be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.

[0122] Figure 7A block diagram schematically illustrates an electronic device suitable for implementing a query processing method according to an embodiment of this application.

[0123] like Figure 7 As shown, an electronic device 700 according to an embodiment of this application includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage portion 708 into a random access memory (RAM) 703. The processor 701 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 701 may also include onboard memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of this application.

[0124] RAM 703 stores various programs and data required for the operation of electronic device 700. Processor 701, ROM 702, and RAM 703 are interconnected via bus 704. Processor 701 executes various operations of the method flow according to embodiments of this application by executing programs in ROM 702 and / or RAM 703. It should be noted that programs may also be stored in one or more memories other than ROM 702 and RAM 703. Processor 701 may also execute various operations of the method flow according to embodiments of this application by executing programs stored in one or more memories.

[0125] According to embodiments of this application, the electronic device 700 may further include an input / output (I / O) interface 705, which is also connected to a bus 704. The electronic device 700 may also include one or more of the following components connected to the input / output (I / O) interface 705: an input section 706 including a keyboard, mouse, etc.; an output section 707 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the input / output (I / O) interface 705 as needed. A removable medium 711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 710 as needed so that computer programs read from it can be installed into the storage section 708 as needed.

[0126] This application also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of this application.

[0127] According to embodiments of this application, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used or combined with an instruction-modified execution system, apparatus, or device. For example, according to embodiments of this application, the computer-readable storage medium may include ROM 702 and / or RAM 703 and / or one or more memories other than ROM 702 and RAM 703 described above.

[0128] Embodiments of this application also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to enable the computer system to implement the query processing method provided in the embodiments of this application.

[0129] When the computer program is executed by the processor 701, it performs the functions defined in the system / apparatus of this application embodiment. According to the embodiments of this application, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0130] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 709, and / or installed from a removable medium 711. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0131] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 709, and / or installed from the removable medium 711. When the computer program is executed by the processor 701, it performs the functions defined in the system of this application embodiment. According to the embodiments of this application, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0132] According to embodiments of this application, program code for executing the computer programs provided in the embodiments of this application can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0133] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0134] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application.

Claims

1. A query processing method, characterized in that, The method includes: Obtain the original query text and preprocess it; The preprocessed original query text is input into the pre-trained target model to obtain the entity of the original query text; The entities in the original query text are standardized. The standardized entities are expanded and reconstructed to obtain an executable query representation. Based on the intent of the executable query representation, the target query template is determined, and the executable query representation is filled into the placeholders of the target query template to obtain the target query statement for retrieval. The target model is a dual-architecture model that includes a generation path and a compatibility path.

2. The query processing method according to claim 1, characterized in that, The process of obtaining and preprocessing the original query text includes: The original query text is cleaned and standardized. A language model based on pseudo-log-likelihood probability is used to determine target candidate words in a pre-defined candidate word set. Based on the target candidate words, the original query text after normalization is corrected for typos.

3. The query processing method according to claim 1, characterized in that, Before inputting the preprocessed original query text into the pre-trained target model, the following steps are also included: The original query text is constructed into multiple text segments; After inserting an identifier before each text segment, the text segment is segmented based on a preset sliding window.

4. The query processing method according to claim 1, characterized in that, The target model is a dual-architecture model comprising a generative model and a discriminative model. The generative model executes the generative pathway, and the discriminative model executes the compatibility pathway. The generative model includes a sequence-to-sequence generator, and the discriminative model is a discriminative model based on a bidirectional encoder and a conditional random field.

5. The query processing method according to claim 4, characterized in that, The target model is pre-trained, the sequence-to-sequence generator includes an encoder and a decoder, and the training process includes: The encoder receives multiple text segments to be trained and generates context vectors; Based on the constraint generation and copying mechanism, the decoder decodes the context vector to obtain the entity of the text to be trained.

6. The query processing method according to claim 5, characterized in that, The target model is pre-trained, and the training process further includes: When the confidence of the generated path is insufficient, the text segment to be trained is trained based on the discriminant of the bidirectional encoder and the conditional random field to obtain the entity of the text to be trained.

7. The query processing method according to claim 1, characterized in that, The standardization of entities in the original query text includes: Determine the type of the entity based on its label; Based on the type of the entity, standard normalization is performed respectively. The types of the entity include time entity, name entity, and amount entity.

8. The query processing method according to claim 1, characterized in that, The process of query expansion and reconstruction of the standardized entities to obtain an executable query representation includes: Establish a candidate set of maps; Based on a preset expansion discriminator, the candidate set of the atlas is discriminated to determine the candidate set of the atlas that can be used for expansion; Based on the graph candidates that can be used for expansion, the standardized entities are query expanded and reconstructed to obtain an executable query representation.

9. A query processing device, characterized in that, The device includes: The raw query text acquisition module is used to acquire the raw query text and perform preprocessing. The entity acquisition module is used to input the preprocessed original query text into the pre-trained target model to obtain the entities of the original query text; The standard normalization processing module is used to perform standard normalization on the entities of the original query text; The executable query representation acquisition module is used to perform query expansion and reconstruction on the standardized normalized entities to obtain executable query representations. The target query statement acquisition module is used to fill the placeholders of the target query template with the executable query representation based on the intent of the executable query representation, so as to obtain the target query statement for retrieval. The target model is a dual-architecture model that includes a generation path and a compatibility path.

10. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 8.

12. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 8.

Images