Information retrieval method and device, related equipment and computer program product
By integrating multiple expert models with a dynamic routing network, the method addresses the limitations of single models in handling complex and diverse data, enhancing retrieval accuracy and adaptability across different scenarios.
Patent Information
- Application Number
- CN202411351015.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-26
- Publication Date
- 2025-01-03
AI Technical Summary
Traditional information retrieval methods struggle with complex and diverse data, leading to inaccurate retrieval results and limited adaptability across various application scenarios due to reliance on single models that perform poorly in cross-domain and multi-type environments.
An information retrieval method that integrates multiple expert models from different fields, utilizing a dynamic routing network to dynamically allocate output weights and merge retrieval results, enhancing the system's adaptability and accuracy through ensemble learning.
The method improves retrieval accuracy and adaptability by leveraging the strengths of multiple expert models, effectively handling complex queries and diverse data types, enabling a more general and open information retrieval system.
Smart Images
Figure CN119248834A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of information retrieval technology, and more specifically, to an information retrieval method, apparatus, related equipment and computer program product. Background Art
[0002] With the rapid development of the Internet, the amount of information has increased dramatically. How to efficiently retrieve the required information from massive data has become an important research topic. Traditional information retrieval methods mostly rely on a single model, which often shows limitations when facing complex and diverse information data, resulting in inaccurate retrieval results and difficulty meeting the needs of a wide range of application scenarios. Summary of the invention
[0003] In view of the above problems, the present application is proposed to provide an information retrieval method, apparatus, related equipment and computer program product to improve the accuracy of retrieval results for complex and diverse information data.
[0004] The specific plan is as follows:
[0005] In a first aspect, an information retrieval method is provided, comprising:
[0006] Get the user's query request;
[0007] Determining the output weight of each expert model in the expert model set according to the query request through a configured dynamic routing network, wherein the expert model set includes more than two expert models for performing information retrieval tasks, and different expert models correspond to different retrieval fields;
[0008] Calling each expert model in the expert model set, performing a search in a configured information retrieval library based on the query request, and obtaining a search result of each expert model;
[0009] According to the output weights of each expert model, the retrieval results of each expert model are merged to obtain the final retrieval result.
[0010] In a possible design, in another implementation of the first aspect of the embodiments of the present application, the process of determining the output weight of each expert model in the expert model set according to the query request through the configured dynamic routing network includes:
[0011] The feature representation of the query request is obtained through the configured dynamic routing network, and the output weight of each expert model in the expert model set is determined based on the feature representation, wherein the feature representation of the query request includes features of multiple different scales.
[0012] In a possible design, in another implementation of the first aspect of the embodiments of the present application, the process of determining the output weight of each expert model in the expert model set based on the feature representation includes:
[0013] Features of different scales in the feature representation are merged, and a 1×1 convolution is used to perform cross-scale feature fusion processing on the merged features, and the output weight of each expert model in the expert model set is determined based on the fused features.
[0014] In a possible design, in another implementation manner of the first aspect of the embodiments of the present application, before determining the output weight of each expert model in the expert model set based on the feature representation, the method further includes:
[0015] Performing attention mechanism processing on the features of each scale in the feature representation respectively to obtain attention enhanced features of each scale;
[0016] The process of determining the output weight of each expert model in the expert model set based on the feature representation includes:
[0017] The output weight of each expert model in the expert model set is determined based on the attention-enhanced features at different scales.
[0018] In a possible design, in another implementation manner of the first aspect of the embodiments of the present application, the query request is in text form, and the features at different scales include local features and global features of the query request text.
[0019] In a possible design, in another implementation of the first aspect of the embodiment of the present application, the dynamic routing network and each expert model in the expert model set constitute an integrated model, the integrated model is pre-trained using labeled data, and the parameters of each expert model are frozen during the training process, and only the parameters of the dynamic routing network are updated;
[0020] The annotated data includes query request samples annotated with search result tags.
[0021] In a possible design, in another implementation of the first aspect of the embodiment of the present application, the process of training the integrated model using labeled data includes:
[0022] Get query request samples;
[0023] Acquire a feature representation of the query request sample through a dynamic routing network, and determine an output weight of each expert model in the expert model set based on the feature representation;
[0024] Sending the feature representation of the query request sample and the feature representation of each piece of information in the configured information retrieval library to each expert model to obtain a retrieval result of each expert model;
[0025] According to the output weights of each expert model, the search results of each expert model are merged to obtain a merged search result;
[0026] A loss value is determined according to the merged retrieval result and the retrieval result label corresponding to the query request sample, and the parameters of the dynamic routing network are updated according to the loss value until the set training end condition is reached.
[0027] In one possible design, in another implementation of the first aspect of the embodiment of the present application, each expert model in the expert model set is trained using pre-training data of the corresponding retrieval field, and the pre-training data includes query request samples under the corresponding retrieval field, and retrieval result labels of the query request samples.
[0028] In a second aspect, an information retrieval device is provided, comprising:
[0029] A query request obtaining unit, used to obtain a user's query request;
[0030] A dynamic routing unit, configured to determine, through a configured dynamic routing network, an output weight of each expert model in an expert model set according to the query request, wherein the expert model set includes more than two expert models for performing information retrieval tasks, and different expert models correspond to different retrieval fields;
[0031] An expert model retrieval unit, used to call each expert model in the expert model set, perform retrieval in a configured information retrieval library based on the query request, and obtain a retrieval result of each expert model;
[0032] The retrieval result merging unit is used to merge the retrieval results of each expert model according to the output weight of each expert model to obtain the final retrieval result.
[0033] In a third aspect, an electronic device is provided, comprising: a memory and a processor;
[0034] The memory is used to store programs;
[0035] The processor is used to execute the program to implement the information retrieval method described in any one of the first aspects of the present application.
[0036] In a fourth aspect, a readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the information retrieval method described in any one of the first aspects of the present application is implemented.
[0037] In a fifth aspect, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the information retrieval method described in any one of the aforementioned first aspects of the present application.
[0038] By means of the above technical scheme, the information retrieval method provided by the present application configures an expert model set, which includes more than two expert models. Different expert models correspond to different retrieval fields and are better at handling information retrieval tasks in corresponding fields. On this basis, for the user's query request, the output weight of each expert model in the expert model set is dynamically allocated through the configured dynamic routing network. By calling each expert model, a search is performed in the configured information retrieval library based on the query request to obtain the retrieval result of each expert model, and the retrieval results of each expert model are merged according to the determined output weights of each expert model to obtain the final retrieval result. The present application integrates expert models from multiple different fields, and dynamically allocates the output weights of each expert model for the current query request through a dynamic routing network, providing a more powerful integrated learning framework, achieving higher retrieval accuracy and a wider scope of application, and improving the accuracy of retrieval results for complex and diverse information data. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:
[0040] Figure 1 A schematic diagram of an implementation system architecture of the information retrieval method provided in an embodiment of the present application;
[0041] Figure 2 A schematic diagram of a terminal structure provided in an embodiment of the present application;
[0042] Figure 3 A schematic diagram of a server structure provided in an embodiment of the present application;
[0043] Figure 4 A schematic diagram of a flow chart of an information retrieval method provided in an embodiment of the present application;
[0044] Figure 5 A schematic diagram of the structure of an expert model provided in an embodiment of the present application;
[0045] Figure 6 A schematic diagram of an information retrieval process of an integrated model composed of a dynamic routing network and expert models provided in an embodiment of the present application;
[0046] Figure 7 A schematic diagram of a process of enhancing the attention mechanism of different scale features and determining the output weights of each expert model in a dynamic routing network provided in an embodiment of the present application;
[0047] Figure 8 A schematic diagram of the structure of an information retrieval device provided in an embodiment of the present application;
[0048] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0049] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0050] The present application provides an information retrieval method that can be applied to retrieval scenarios of various types of information, such as text information retrieval scenarios, image information retrieval scenarios, video information retrieval scenarios, etc. The information retrieval method of the present application can be applied to Figure 1 The system architecture shown in FIG. 1 may include a terminal 100 and a server 200. The server 200 may include one or more servers ( Figure 1 A server is included as an example for explanation).
[0051] The terminal 100 or the server 200 can be used alone to execute the information retrieval method provided in the embodiment of the present application. In addition, the terminal 100 and the server 200 can also be used together to execute the information retrieval method provided in the embodiment of the present application.
[0052] Next describe Figure 1 The product form of the mid-terminal 100;
[0053] The terminal 100 in the embodiment of the present application can be a mobile phone, a tablet computer, a teaching screen, a wearable device, a vehicle-mounted device, a conference terminal, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc., and the embodiment of the present application does not impose any restrictions on this.
[0054] Figure 2 An optional hardware structure diagram of the terminal 100 is shown.
[0055] refer to Figure 2 As shown, the terminal 100 may include a radio frequency unit 110, a memory 120, an input unit 130, a display unit 140, a camera 150 (optional), an audio circuit 160, a speaker 161, a microphone 162, an earphone jack 163 (optional), a processor 170, an external interface 180, a power supply 190 and other components. Those skilled in the art will appreciate that Figure 2 These are merely examples of terminals or multi-function devices and do not constitute limitations on the terminals or multi-function devices, which may include more or fewer components than those shown in the figures, or combinations of certain components, or different components.
[0056] The input unit 130 can be used to receive input digital or character information, and generate key signal input related to the user settings and function control of the portable multifunctional device. Specifically, the input unit 130 may include a touch screen 131 and / or other input devices 132. The touch screen 131 can collect the user's touch operations on or near it (such as the user's operation on or near the touch screen using any suitable object such as fingers, joints, stylus, etc.), and drive the corresponding connection device according to a pre-set program. The touch screen can detect the user's touch action on the touch screen, convert the touch action into a touch signal and send it to the processor 170, and can receive and execute the command sent by the processor 170; the touch signal at least includes the touch point coordinate information. The touch screen 131 can provide an input interface and an output interface between the terminal 100 and the user. In addition, the touch screen can be implemented using multiple types such as resistive, capacitive, infrared and surface acoustic wave. In addition to the touch screen 131, the input unit 130 can also include other input devices. Specifically, other input devices 132 may include, but are not limited to, one or more of a physical keyboard, function keys (such as a volume control key, a switch key, etc.), a trackball, a mouse, a joystick, and the like.
[0057] Among them, other input devices 132 can receive input data and so on.
[0058] The display unit 140 can be used to display information input by the user or information provided to the user, various menus of the terminal 100, interactive interfaces, file display and / or playback of any multimedia file. In the embodiment of the present application, the display unit 140 can be used to display various interactive interfaces, processing results, etc. in the information retrieval method.
[0059] The memory 120 can be used to store instructions and data. The memory 120 can mainly include an instruction storage area and a data storage area. The data storage area can store various data, such as multimedia files, texts, etc.; the instruction storage area can store software units such as operating systems, applications, instructions required for at least one function, or their subsets and extensions. It can also include a non-volatile random access memory; provide the processor 170 with hardware, software and data resources including management of computing and processing equipment, and support control software and applications. It is also used for the storage of multimedia files, and the storage of running programs and applications.
[0060] The processor 170 is the control center of the terminal 100. It uses various interfaces and lines to connect various parts of the entire terminal 100. By running or executing instructions stored in the memory 120 and calling data stored in the memory 120, it executes various functions of the terminal 100 and processes data, thereby controlling the terminal device as a whole. Optionally, the processor 170 may include one or more processing units; preferably, the processor 170 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface and application program, and the modem processor mainly processes wireless communication. It is understandable that the above-mentioned modem processor may not be integrated into the processor 170. In some embodiments, the processor and the memory may be implemented on a single chip, and in some embodiments, they may also be implemented separately on separate chips. The processor 170 may also be used to generate corresponding operation control signals, send them to corresponding components of the computing and processing device, read and process data in the software, especially read and process data and programs in the memory 120, so that each functional module therein performs corresponding functions, thereby controlling the corresponding components to act according to the requirements of the instructions.
[0061] Among them, the memory 120 can be used to store software codes related to the information retrieval method, the processor 170 can execute the steps of the information retrieval method, and can also schedule other units (such as the above-mentioned input unit 130 and display unit 140) to achieve corresponding functions.
[0062] The radio frequency unit 110 (optional) can be used for receiving and sending information or receiving and sending signals during a call, for example, after receiving the downlink information of the base station, it is sent to the processor 170 for processing; in addition, the designed uplink data is sent to the base station. Generally, the RF circuit includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (Low Noise Amplifier, LNA), a duplexer, etc. In addition, the radio frequency unit 110 can also communicate with network devices and other devices through wireless communication. The wireless communication can use any communication standard or protocol, including but not limited to Global System of Mobile communication (Global System of Mobile communication, GSM), General Packet Radio Service (General Packet Radio Service, GPRS), Code Division Multiple Access (Code Division Multiple Access, CDMA), Wideband Code Division Multiple Access (Wideband Code Division Multiple Access, WCDMA), Long Term Evolution (Long Term Evolution, LTE), email, Short Messaging Service (SMS), etc.
[0063] In the embodiment of the present application, the RF unit 110 can send data to the server 200 and receive the processing result sent by the server 200. Exemplarily, the RF unit 110 sends the received user's query request to the server 200, and the server 200 obtains the search result corresponding to the query request and returns the search result to the terminal 100.
[0064] It should be understood that the radio frequency unit 110 is optional and can be replaced by other communication interfaces, such as a network port.
[0065] The terminal 100 also includes a power supply 190 (such as a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the processor 170 through a power management system, so that the power management system can manage functions such as charging, discharging, and power consumption.
[0066] The terminal 100 further includes an external interface 180 , which may be a standard Micro USB interface or a multi-pin connector, and may be used to connect the terminal 100 to communicate with other devices, or to connect a charger to charge the terminal 100 .
[0067] Although not shown, the terminal 100 may also include a flashlight, a wireless fidelity (WiFi) module, a Bluetooth module, sensors with different functions, etc., which are not described in detail here. Some or all of the methods described below may be applied in the following embodiments. Figure 2 In the terminal 100 shown.
[0068] Next describe Figure 1 The product form of the server 200;
[0069] Figure 3 A structural diagram of a server 200 is provided, such as Figure 3 As shown, the server 200 includes a bus 201, a processor 202, a communication interface 203, and a memory 204. The processor 202, the memory 204, and the communication interface 203 communicate with each other via the bus 201.
[0070] The bus 201 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0071] The processor 202 may be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).
[0072] The memory 204 may include a volatile memory, such as a random access memory (RAM). The memory 204 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).
[0073] The memory 204 may be used to store software codes related to the information retrieval method, and the processor 202 may execute the steps of the information retrieval method and may also schedule other units to implement corresponding functions.
[0074] It should be understood that the above-mentioned terminal 100 and server 200 can be centralized or distributed devices, and the processors in the above-mentioned terminal 100 and server 200 (such as processor 170 and processor 202) can be hardware circuits (such as application specific integrated circuit (ASIC), field-programmable gate array (FPGA), general-purpose processor, digital signal processor (DSP), microprocessor or microcontroller, etc.), or a combination of these hardware circuits. For example, the processor can be a hardware system with an instruction execution function, such as a CPU, DSP, etc., or a hardware system without an instruction execution function, such as an ASIC, FPGA, etc., or a combination of the above-mentioned hardware systems without an instruction execution function and hardware systems with an instruction execution function.
[0075] In recent years, machine learning and deep learning technologies have made significant progress in the field of information retrieval. By training large language models, better retrieval results can be achieved in specific fields. However, these models usually perform poorly when faced with cross-domain and multi-type data, and there are several major problems:
[0076] Limitations of a single model:
[0077] Although existing technologies use powerful language models such as BERT and Sentence-BERT, which show excellent performance in specific datasets or fields, a single model may perform poorly when processing different types of queries and information data, especially in cross-domain or diverse language environments, where its retrieval accuracy and adaptability are usually limited.
[0078] Insufficient generalization ability:
[0079] Existing text retrieval models rely on specific datasets and contexts in training and application, which limits the generalization ability of the models. When faced with new or unseen information types and queries, the performance of the models often drops significantly, which is particularly evident in open-world information retrieval applications.
[0080] Inadequate handling of complex queries:
[0081] Existing retrieval models often fail to effectively utilize multiple key information points in a query when dealing with complex queries that contain multiple concepts or require in-depth understanding. A single model may have difficulty extracting all relevant semantic information from a complex query, resulting in inaccurate or less relevant retrieval results.
[0082] In order to overcome the above problems, this application proposes an information retrieval method based on ensemble learning. This method integrates information retrieval expert models from different fields and utilizes the advantages of each model in a specific field to build a general information retrieval system with high generalization ability.
[0083] Next, the information retrieval method provided in the embodiment of the present application is introduced, and the method is applied to Figure 1 Taking the computer device in the example as an example, the computer device can be Figure 1 The terminal 100 or the system consisting of the terminal 100 and the server 200. Figure 4 , the information retrieval method specifically comprises the following steps:
[0084] Step S100: Obtain the user's query request.
[0085] Specifically, the information retrieval method of the present application can be applied to a variety of information retrieval scenarios, such as text information retrieval scenarios, image information retrieval scenarios, video information retrieval scenarios, etc. The present application can be pre-configured with an information retrieval library, in which information to be retrieved is stored, such as documents, images, videos, etc. When a user has a retrieval demand, he can initiate a query request, which can be in text form, or in voice or other modal requests.
[0086] In the embodiment of the present application, the text search scenario is taken as an example. In order to be applicable to text searches in various fields, text information in multiple fields can be collected in advance, such as texts in the fields of science and technology, medicine, law, education, etc., and stored in the information search library to ensure that the data covers a wide range and can support a variety of query requests.
[0087] The query request obtained may be a request of multiple modes. In this application, the subsequent steps may be directly performed based on the query request of the original mode obtained. In addition, the query request of the original mode obtained may be converted into a set mode, such as uniformly converting the query request into a text mode, for use in subsequent steps.
[0088] Step S110: Determine the output weight of each expert model in the expert model set according to the query request through the configured dynamic routing network.
[0089] The expert model set includes more than two expert models for performing information retrieval tasks, and different expert models correspond to different retrieval fields.
[0090] In this embodiment, multiple expert models can be defined according to the retrieval requirements. Each expert model focuses on the deep semantic understanding of a specific field, and the information retrieval accuracy in the specific field is higher. For example: financial field retrieval expert model, automobile field retrieval expert model, education field retrieval expert model, medical field retrieval expert model, etc., each of which is good at information retrieval tasks in the corresponding field.
[0091] It should be noted here that the expert model corresponding to a certain search field is not only capable of performing information retrieval tasks in the corresponding search field, but can also perform information retrieval tasks in other search fields, but the retrieval accuracy in the corresponding search field is relatively higher.
[0092] Reference Figure 5 , which illustrates a structural diagram of an expert model.
[0093] Taking the text retrieval task as an example, the model can use BERT (or other pre-trained language models) to extract the embedding vector of the input text, including extracting the embedding vector of the query request query, and extracting the embedding vector of each document in the text retrieval library.
[0094] It is understandable that for information retrieval tasks in other modalities, such as information retrieval tasks in image and audio modalities, the model can use other types of models to extract embedded vectors of images and audios, which will not be explained in detail here.
[0095] Furthermore, the model performs a pooling operation on the embedded vectors of the extracted text to obtain the query vector u corresponding to the query and the vector representation v of each document in the text retrieval library.
[0096] Calculate the similarity between the query vector u and the vector representation v of each document in the text retrieval library, such as using cosine similarity, and then determine the target document as the final retrieval result from the text retrieval library according to the similarity.
[0097] For each expert model in the expert model set, the pre-training data in the corresponding retrieval field can be used for preliminary training. The pre-training data includes the query request samples in the corresponding retrieval field and the retrieval result labels of the query request samples. The classification objective function can be used to fine-tune the BERT model during the training process. Detailed parameter examples include: batch-size of 16, Adam optimizer, optimezer learning rate of 2e-5, 10% linear learning rate warmup, and average pooling.
[0098] Furthermore, a dynamic routing network is also configured in this embodiment, which can realize dynamic routing through the network layer, that is, dynamically adjust the output weight of each expert model according to the user's query request, thereby replacing the traditional multi-model voting and scoring mechanism, which can be better adapted to query requests in different scenarios and realize cross-scenario university retrieval.
[0099] In a possible implementation, the dynamic routing network and each expert model in the expert model set constitute an integrated model, which is pre-trained with labeled data, and the parameters of each expert model are frozen during the training process, and only the parameters of the dynamic routing network are updated, so that the dynamic routing network learns how to optimize the selection of expert models, that is, to determine the output weights of different expert models, so that the final retrieval results are more accurate. Among them, the labeled data includes query request samples labeled with retrieval result labels.
[0100] Step S120: call each expert model in the expert model set, perform retrieval in the configured information retrieval library based on the query request, and obtain the retrieval result of each expert model.
[0101] Specifically, each expert model is called separately, and a retrieval task is performed in the information retrieval library based on the input query request, and the retrieval result of each expert model can be obtained. The retrieval result includes the probability distribution of each information in the information retrieval library belonging to the retrieval result corresponding to the query request.
[0102] Step S130: merge the search results of the expert models according to the output weights of the expert models to obtain the final search results.
[0103] Specifically, the output weight of each expert model is obtained in the aforementioned step S110, and the output weight represents the credibility of the retrieval result of the corresponding expert model as the final retrieval result. In this step, in order to fuse the retrieval results of different expert models, the retrieval results of each expert model can be merged according to the output weight of each expert model, such as weighted addition of the probability distribution output by each expert model according to the output weight to obtain the final probability distribution. Furthermore, based on the final probability distribution, one or topN pieces of information with the largest probability value can be selected as the final retrieval result output.
[0104] The final search result y can be expressed as the following formula:
[0105]
[0106] Among them, the expert model set contains n experts, Fusion_Feature(x) k represents the output weight of the kth expert model output by the dynamic routing network, E k(x) represents the retrieval result of the kth expert model, and x represents the query request query entered by the user.
[0107] The information retrieval method provided in the embodiment of the present application is configured with an expert model set, which includes more than two expert models. Different expert models correspond to different retrieval fields and are better at processing information retrieval tasks in corresponding fields. On this basis, for the user's query request, the output weight of each expert model in the expert model set is dynamically allocated through the configured dynamic routing network. By calling each expert model, a search is performed in the configured information retrieval library based on the query request to obtain the retrieval result of each expert model, and the retrieval results of each expert model are merged according to the determined output weights of each expert model to obtain the final retrieval result. The present application integrates expert models from multiple different fields, and dynamically allocates the output weights of each expert model for the current query request through a dynamic routing network, providing a more powerful integrated learning framework, achieving higher retrieval accuracy and a wider scope of application, and improving the accuracy of retrieval results for complex and diverse information data.
[0108] The method proposed in this application not only effectively solves the problem of insufficient generalization ability of a single model in information retrieval, but also fully utilizes the advantages of expert models in different fields through ensemble learning, realizes efficient retrieval of cross-domain and multi-type information data, and provides technical support for building a more general and open information retrieval system.
[0109] Compared with the various problems existing in the existing retrieval methods mentioned above, this application solves them through an integrated learning method, which has the following advantages:
[0110] Integration of multi-model advantages: By integrating multiple expert models from different fields, this application builds a more powerful integrated learning framework to achieve higher retrieval accuracy and wider scope of application. This approach allows the system to combine the advantages of different expert models and provide more comprehensive information understanding and retrieval capabilities.
[0111] Enhance the generalization ability of the model: Integrating multiple expert models enables the system to better adapt to new data types and retrieval requirements, thereby improving generalization capabilities. This is especially important for coping with various retrieval challenges in the open world, and can effectively improve the adaptability and flexibility of the system.
[0112] Improved complex query processing: Through ensemble learning, the system can more effectively process queries with complex semantics because it can obtain and integrate richer information from multiple expert models, thereby improving the understanding and response capabilities of complex queries.
[0113] Combination Figure 6, which illustrates a schematic diagram of the information retrieval process of an integrated model consisting of a dynamic routing network and various expert models.
[0114] The query request entered by the user is represented as query.
[0115] Send the query request query to the dynamic routing network, and you can get the output weight Fusion_Feature(x) of each expert model output by the dynamic routing network. k The query request query sent to the dynamic routing network may be an embedded vector representation, such as an embedded vector representation converted by a word embedding unit.
[0116] Send the query request and each piece of information in the retrieval database to each expert model in the expert model set (different expert models are represented by E 1-n (x)), and obtain the retrieval results output by each expert model, which can be the vector similarity probability distribution between the query request query and each piece of information in the retrieval library. Among them, the query request query sent to the expert model and each piece of information in the retrieval library can be an embedded vector representation, such as obtained after conversion through a word embedding unit.
[0117] The retrieval results of each expert model are weighted and added according to the corresponding output weights to obtain the final retrieval result. For example, the one or top N pieces of information with the largest probability value are selected from the probability distribution after weighted addition as the final retrieval result.
[0118] In some embodiments of the present application, the training process of the integrated model composed of the dynamic routing network and the expert models is described, which may include the following steps:
[0119] S1. Obtain query request samples.
[0120] Specifically, before training the integrated model, training data is prepared, including query request samples and corresponding retrieval result labels.
[0121] S2. Obtain feature representation of the query request sample through a dynamic routing network, and determine the output weight of each expert model in the expert model set based on the feature representation.
[0122] S3. Send the feature representation of the query request sample and the feature representation of each piece of information in the configured information retrieval library to each expert model to obtain the retrieval result of each expert model.
[0123] S4. According to the output weights of the expert models, the search results of the expert models are merged to obtain a merged search result.
[0124] S5. Determine the loss value according to the merged retrieval results and the retrieval result label corresponding to the query request sample, and update the parameters of the dynamic routing network according to the loss value until the set training end condition is reached.
[0125] The loss function can be a classification loss function. During the training process of the integrated model, the parameters of each expert model are frozen, and only the parameters of the dynamic routing network are updated. After the training end condition is reached, the integrated model can be deployed to the task environment to perform the retrieval task.
[0126] In some embodiments of the present application, the process of determining the output weight of each expert model in the expert model set according to the query request through the configured dynamic routing network in step S110 in the aforementioned embodiment is described.
[0127] This embodiment provides a multi-scale feature extraction strategy, which can extract local and global scale features of query requests. Specifically:
[0128] A feature representation of a query request may be obtained through a configured dynamic routing network, and an output weight of each expert model in an expert model set may be determined based on the feature representation, wherein the feature representation of the query request includes features of multiple different scales.
[0129] Taking a query request in text form as an example, the feature representation of the query request may include local features and global features, such as: word-level and phrase-level local features, and sentence-level global features.
[0130] In this embodiment, features of different scales in the feature representation of the query request may be merged, and the merged features may be fused, and the output weight of each expert model in the expert model set may be determined based on the fused features.
[0131] In one possible implementation, when merging features, Conv1*1 (1Δ×1 convolution method) can be used to perform cross-scale feature fusion processing on features of different scales, replacing the traditional softmax voting scoring mechanism method. By using Conv1*1, features of different scales can be more effectively fused, enhancing the characteristics of multi-scale feature crossover.
[0132] Further optionally, before determining the output weight of each expert model in the expert model set based on the feature representation of the query request, a step of performing an attention mechanism enhancement process on features of different scales may be added, that is:
[0133] The attention mechanism is performed on the features of each scale in the feature representation of the query request to obtain the attention enhanced features of each scale.
[0134] Specifically, the attention mechanism is applied to features of different scales respectively, which can effectively refine the local feature representation of the query request, capture contextual dependencies and important information, and thus achieve feature enhancement.
[0135] The formula for attention mechanism processing can be expressed as follows:
[0136] Q i =W Q X
[0137] K i =W K X
[0138] V i =W V X
[0139]
[0140] Where i represents different feature scales. Taking text query requests as an example, feature scales can include word level, phrase level, sentence level, etc., i∈(word, phrase, sentence), X represents the feature representation of the query request, and its dimension is (batc_h si,ze sequ_ence,length_hi)d, where hidden_size is the hidden layer dimension, Q, K, V represent query, key, and value attention matrices respectively, and W Q , W K , W V They are the weight matrices of query, key, and value, with dimensions of (hidden_size, hidden_size), Attention i Represents the attention-enhanced features at each scale.
[0141] Combination Figure 7 , which takes text query requests as an example to introduce the dynamic routing network for different scales ( Figure 7 Schematic diagram of the process of enhancing the attention mechanism based on features (taking word level, phrase level, and sentence level as examples) and determining the output weights of each expert model.
[0142] For the feature representation X of the query request, the attention matrix and weight matrix at the word level, phrase level, and sentence level are calculated respectively, and then according to the above Attention i The calculation formula calculates the attention enhancement features of each scale: Attention word 、Attention phrase 、Attention sentence .
[0143] In this embodiment, the traditional single self-attention method is expanded to Attention i , reflecting multi-scale attention, which can effectively capture features of different scales, such as word, phrase-level local features and sentence-level global features, providing more accurate and in-depth query responses.
[0144] Furthermore, the attention enhancement features of each scale are merged and concat, and the merged features are fused using Conv11. The output weight of each expert model in the expert model set is determined based on the fused features. Fusion_Feature(x) k :
[0145] Fusion_Feature=Conv1*1(concat(Attention i ))
[0146] From the introduction of the above embodiments, it can be seen that the present application proposes an information retrieval method based on ensemble learning, which mixes and integrates multiple independently trained expert models for information retrieval into a framework. This strategy allows the system to significantly improve its ability to handle complex and diverse information retrieval across scenarios.
[0147] In addition, the present application provides a dynamic routing weight method, in which the dynamic routing network adopts Conv1*1 to replace the traditional softmax voting scoring mechanism, effectively enhancing the characteristics of multi-scale feature crossover, so that the entire integrated model can dynamically adjust the output weights of different expert models according to the query request query input by the user in real time.
[0148] Furthermore, the present application also provides a multi-scale feature extraction and enhancement method, which extracts multi-scale features from query requests through a dynamic routing network and expands the traditional single Attention method to Attention i , where i takes values at different scales, such as i∈(word, phrase, sentence), which reflects multi-scale attention and can effectively capture features at different scales, such as local features at the word and phrase level and global features at the sentence level, providing more accurate and in-depth query responses.
[0149] The information retrieval device provided in the embodiment of the present application is described below. The information retrieval device described below and the information retrieval method described above can be referenced to each other.
[0150] See also Figure 8 , Figure 8 A schematic diagram of the structure of an information retrieval device disclosed in an embodiment of the present application.
[0151] like Figure 8 As shown, the device may include:
[0152] A query request obtaining unit 11, used to obtain a user's query request;
[0153] A dynamic routing unit 12, configured to determine the output weight of each expert model in the expert model set according to the query request through a configured dynamic routing network, wherein the expert model set includes more than two expert models for performing information retrieval tasks, and different expert models correspond to different retrieval fields;
[0154] An expert model retrieval unit 13, configured to call each expert model in the expert model set, perform retrieval in a configured information retrieval library based on the query request, and obtain a retrieval result of each expert model;
[0155] The search result merging unit 14 is used to merge the search results of each expert model according to the output weight of each expert model to obtain the final search result.
[0156] In a possible implementation, the process of determining the output weight of each expert model in the expert model set according to the query request by the dynamic routing unit through the configured dynamic routing network includes:
[0157] The feature representation of the query request is obtained through the configured dynamic routing network, and the output weight of each expert model in the expert model set is determined based on the feature representation, wherein the feature representation of the query request includes features of multiple different scales.
[0158] In a possible implementation, the process of the dynamic routing unit determining the output weight of each expert model in the expert model set based on the feature representation includes:
[0159] Features of different scales in the feature representation are merged, and a 1×1 convolution is used to perform cross-scale feature fusion processing on the merged features, and the output weight of each expert model in the expert model set is determined based on the fused features.
[0160] In a possible implementation, before the dynamic routing unit determines the output weight of each expert model in the expert model set based on the feature representation, it further includes:
[0161] The attention mechanism is respectively processed for the features of each scale in the feature representation to obtain the attention enhanced features of each scale. On this basis, the dynamic routing unit determines the output weight of each expert model in the expert model set based on the feature representation, including:
[0162] The output weight of each expert model in the expert model set is determined based on the attention-enhanced features at different scales.
[0163] In a possible implementation, the query request may be in a variety of different modalities. Taking the query request in text form as an example, the features at different scales include local features and global features of the query request text.
[0164] In a possible implementation, the dynamic routing network and each expert model in the expert model set constitute an integrated model, the integrated model is pre-trained using labeled data, and the parameters of each expert model are frozen during the training process, and only the parameters of the dynamic routing network are updated;
[0165] The annotated data includes query request samples annotated with search result tags.
[0166] In a possible implementation, the device of the present application may further include:
[0167] The model training unit is used to train the integrated model using labeled data. The process includes:
[0168] Get query request samples;
[0169] Acquire a feature representation of the query request sample through a dynamic routing network, and determine an output weight of each expert model in the expert model set based on the feature representation;
[0170] Sending the feature representation of the query request sample and the feature representation of each piece of information in the configured information retrieval library to each expert model to obtain a retrieval result of each expert model;
[0171] According to the output weights of each expert model, the search results of each expert model are merged to obtain a merged search result;
[0172] A loss value is determined according to the merged retrieval result and the retrieval result label corresponding to the query request sample, and the parameters of the dynamic routing network are updated according to the loss value until the set training end condition is reached.
[0173] In a possible implementation, each expert model in the expert model set is trained using pre-training data of a corresponding retrieval field, and the pre-training data includes query request samples under the corresponding retrieval field and retrieval result labels of the query request samples.
[0174] The present application also provides an electronic device in an embodiment. Fig. 9As shown, it shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiment of the present application. The electronic device in the embodiment of the present application may include but is not limited to fixed terminals such as mobile phones, tablet computers, learning machines, servers, intelligent robots, etc. Fig. 9 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.
[0175] like Fig. 9 As shown, the electronic device may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603 to implement the information retrieval method of the aforementioned embodiment of the present application. When the electronic device is powered on, various programs and data required for the operation of the electronic device are also stored in RAM 603. Processing device 601, ROM 602, and RAM 603 are connected to each other via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.
[0176] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a memory card, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although Fig. 9 An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.
[0177] An embodiment of the present application also provides a computer program product including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any information retrieval method provided in the embodiment of the present application.
[0178] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any information retrieval method provided in an embodiment of the present application.
[0179] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.
[0180] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.
[0181] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.
[0182] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.
[0183] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can refer to each other.
Claims
1. An information retrieval method, characterized in that: include: Get the user's query request; Determining the output weight of each expert model in the expert model set according to the query request through a configured dynamic routing network, wherein the expert model set includes more than two expert models for performing information retrieval tasks, and different expert models correspond to different retrieval fields; Calling each expert model in the expert model set, performing a search in a configured information retrieval library based on the query request, and obtaining a search result of each expert model; According to the output weights of each expert model, the retrieval results of each expert model are merged to obtain the final retrieval result.
2. The method according to claim 1, characterized in that The process of determining the output weight of each expert model in the expert model set according to the query request through the configured dynamic routing network includes: The feature representation of the query request is obtained through the configured dynamic routing network, and the output weight of each expert model in the expert model set is determined based on the feature representation, wherein the feature representation of the query request includes features of multiple different scales.
3. The method according to claim 2, characterized in that The process of determining the output weight of each expert model in the expert model set based on the feature representation includes: Features of different scales in the feature representation are merged, and a 1×1 convolution is used to perform cross-scale feature fusion processing on the merged features, and the output weight of each expert model in the expert model set is determined based on the fused features.
4. The method according to claim 2 or 3, characterized in that: Before determining the output weight of each expert model in the expert model set based on the feature representation, the method further includes: Performing attention mechanism processing on the features of each scale in the feature representation respectively to obtain attention enhanced features of each scale; The process of determining the output weight of each expert model in the expert model set based on the feature representation includes: The output weight of each expert model in the expert model set is determined based on the attention-enhanced features at different scales.
5. The method according to claim 2, characterized in that: The query request is in text form, and the features at different scales include local features and global features of the query request text.
6. The method according to claim 1, characterized in that The dynamic routing network and each expert model in the expert model set constitute an integrated model, the integrated model is pre-trained with labeled data, and the parameters of each expert model are frozen during the training process, and only the parameters of the dynamic routing network are updated; The annotated data includes query request samples annotated with search result tags.
7. The method according to claim 6, characterized in that The process of training the integrated model with labeled data includes: Get query request samples; Acquire a feature representation of the query request sample through a dynamic routing network, and determine an output weight of each expert model in the expert model set based on the feature representation; Sending the feature representation of the query request sample and the feature representation of each piece of information in the configured information retrieval library to each expert model to obtain a retrieval result of each expert model; According to the output weights of each expert model, the search results of each expert model are merged to obtain a merged search result; A loss value is determined according to the merged retrieval result and the retrieval result label corresponding to the query request sample, and the parameters of the dynamic routing network are updated according to the loss value until the set training end condition is reached.
8. The method according to claim 1, characterized in that Each expert model in the expert model set is trained using pre-training data in a corresponding search field, and the pre-training data includes query request samples in the corresponding search field and search result labels of the query request samples.
9. An information retrieval device, characterized in that: include: A query request obtaining unit, used to obtain a user's query request; A dynamic routing unit, configured to determine, through a configured dynamic routing network, an output weight of each expert model in an expert model set according to the query request, wherein the expert model set includes more than two expert models for performing information retrieval tasks, and different expert models correspond to different retrieval fields; An expert model retrieval unit, used to call each expert model in the expert model set, perform retrieval in a configured information retrieval library based on the query request, and obtain a retrieval result of each expert model; The retrieval result merging unit is used to merge the retrieval results of each expert model according to the output weight of each expert model to obtain the final retrieval result.
10. An electronic device, characterized in that: include: Memory and processor; The memory is used to store programs; The processor is used to execute the program to implement each step of the information retrieval method according to any one of claims 1 to 8.
11. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, each step of the information retrieval method according to any one of claims 1 to 8 is implemented.
12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, each step of the information retrieval method according to any one of claims 1 to 8 is implemented.