Method and system for domain adaptation learning of document retrieval models

Abstract

The present disclosure includes a domain adaptive learning method for a document search model, which is performed by at least one processor. The domain adaptive learning method may include the steps of: configuring a query encoder and a document encoder based on weights of a base model; placing a first adapter for semantic learning in the transformer layer of the query encoder and placing a second adapter for distribution alignment in the output after pooling of the query encoder; and updating the parameters of the first adapter and the second adapter while fixing the body weights of the query encoder using a dataset of the first domain.

Description

Technology Field

[0001] The present disclosure relates to a domain-adaptive learning method for a document search model and an information processing system for the same. Background Technology

[0002] Recently, in the field of information retrieval, Dense Retrieval technology—which performs searches by transforming the meaning of text into high-dimensional dense vectors based on pre-trained language models—has been advancing rapidly. Dense Retrieval models generally adopt a dual-encoder structure that maps queries and documents into an embedding space using independent encoders, and extracts the most semantically similar documents by calculating the inner product or cosine similarity between the two vectors. This approach has the advantage of overcoming the limitations of traditional Sparse Retrieval, which relied on surface area matching of words, and enabling the identification of contextual meaning.

[0003] However, when search models trained on general large-scale datasets are applied to domains requiring specialized knowledge, such as law, medicine, and finance, they suffer from a domain transition problem in which search performance rapidly deteriorates because they fail to accurately grasp the specialized terminology or sentence structures unique to those fields. To solve this, domain adaptation learning, which fine-tunes model parameters using a specific domain dataset, is inevitably required.

[0004] Conventional domain adaptation methods typically involve simultaneously updating the weights of both the query encoder and the document encoder. However, this approach incurs significant cost issues from a practical operational perspective. This is because changing the document encoder weights requires regenerating all vector values ​​within a massive existing database of millions of document embedding records, necessitating a large-scale re-indexing process. Consequently, this consumes astronomical computing resources and time whenever a new domain is added or the model is updated, acting as a critical bottleneck that hinders the flexible scalability of models in search service environments where real-time performance is paramount. The problem to be solved

[0005] The present disclosure provides a domain-adaptive learning method for a document search model, an information processing system for the same, and a computer-readable recording medium for storing a computer program to solve the above-mentioned problems. means of solving the problem

[0006] The present disclosure may be implemented in various ways, including a method, an apparatus (system), or a computer program stored on a readable storage medium.

[0007] A domain adaptive learning method for a document search model, performed by at least one processor according to one embodiment of the present disclosure for solving technical problems, comprises the steps of: configuring a query encoder and a document encoder based on the weights of a base model — the weights of the document encoder are frozen so as not to be changed during the learning process — placing a first adapter for semantic learning in the transformer layer of the query encoder and placing a second adapter for distribution alignment at the output end after pooling of the query encoder; and updating the parameters of the first adapter and the second adapter while fixing the body weights of the query encoder using a dataset of the first domain, wherein the second adapter may be configured to operate as an identity function that outputs an input vector without modification at the time when learning by the dataset of the first domain begins.

[0008] According to one embodiment of the present disclosure, a first adapter is trained to reflect the domain semantics of text included in a dataset of a first domain, and a second adapter can be trained to geometrically align a query embedding vector generated through pooling of a query encoder with an embedding space distribution already constructed by a document encoder.

[0009] According to one embodiment of the present disclosure, the second adapter may include a residual linkage structure that adds a non-linear correction value calculated through a multilayer perceptron to the output vector after pooling.

[0010] According to one embodiment of the present disclosure, a multilayer perceptron is composed of a plurality of layers including a nonlinear activation function, wherein the weights of the last layer of the multilayer perceptron may be initialized to a zero matrix.

[0011] According to one embodiment of the present disclosure, the first adapter may be trained by being coupled to at least one of an embedding matrix associated with an input embedding layer of a query encoder, a projection matrix associated with a query, key, value, and output included in a multi-head attention layer of a transformer layer, or a linear layer included in a feed-forward network layer of a transformer layer.

[0012] According to one embodiment of the present disclosure, the first adapter is composed of the product of a low-rank based first matrix and a second matrix arranged in parallel with the original weight matrix of the base model, wherein the first matrix is ​​initialized with a Gaussian distribution and the second matrix is ​​initialized with a zero matrix so as to preserve the output of the base model during the initial training.

[0013] According to one embodiment of the present disclosure, a document embedding database generated in advance through a document encoder may have an update process or a re-indexing process omitted during or after the domain adaptive learning method is performed.

[0014] According to one embodiment of the present disclosure, a domain adaptive learning method may further include the steps of updating parameters of a plurality of third adapters associated with a first adapter while fixing the body weights of a query encoder with different domain datasets, merging the updated parameters of the plurality of third adapters through a weighted average method to generate an integrated adapter adapted to a plurality of domains, and replacing the first adapter with the generated integrated adapter.

[0015] According to one embodiment of the present disclosure, the domain adaptive learning method may further include the steps of obtaining a parameter set of an updated fourth adapter and a fifth adapter while fixing the body weights of the query encoder using a dataset of a second domain, and dynamically loading the parameter set of the fourth adapter and the fifth adapter into a base model.

[0016] According to one embodiment of the present disclosure, the non-linear activation function included in the multilayer perceptron may include either ReLU (Rectified Linear Unit) or GeLU (Gaussian Error Linear Unit).

[0017] According to one embodiment of the present disclosure, an information retrieval method using a query encoder comprising a first adapter and a second adapter, the parameters of which are optimized by a domain adaptive learning method, may include the steps of receiving a query text associated with a first domain; tokenizing the query text and generating a query vector associated with the query text through an input embedding layer of the query encoder; generating a query embedding vector based on the query vector — the query embedding vector reflects the domain semantics of the first domain through the first adapter and is aligned on a document embedding space distribution through the second adapter —, calculating a similarity between the query embedding vector and document vectors in a document embedding database pre-constructed by a document encoder with frozen weights, and extracting and providing a search result based on the calculated similarity.

[0018] According to one embodiment of the present disclosure, the step of generating a query embedding vector may include identifying a domain from a received query text and dynamically loading a pre-trained set of adapter parameters corresponding to the identified domain into a query encoder.

[0019] According to one embodiment of the present disclosure, a domain adaptive learning method may further include the steps of receiving a search request for a second domain different from a first domain, and performing a search by replacing the adapter parameter set of a query encoder with an updated parameter set corresponding to the second domain without a re-indexing process of a document embedding database.

[0020] According to one embodiment of the present disclosure, the step of calculating similarity includes performing a cosine similarity or inner product operation between generated query embedding vectors and document vectors, and the step of providing search results may include extracting a preset number of documents in order of highest calculated similarity and sorting the extracted documents into a list form and providing them.

[0021] According to one embodiment of the present disclosure, an information processing system comprises a memory and a processor connected to the memory and configured to execute at least one computer-readable program contained in the memory, wherein the at least one program comprises a query encoder and a document encoder based on weights of a base model, a first adapter for semantic learning placed in the transformer layer of the query encoder, a second adapter for distribution alignment placed at the output end after pooling of the query encoder, and instructions for updating the parameters of the first adapter and the second adapter while fixing the body weights of the query encoder using a dataset of the first domain, wherein the weights of the document encoder are frozen so as not to be changed during the learning process, and the second adapter may operate as an identity function that outputs the input vector of the second adapter without modification at the time when learning begins using the dataset of the first domain. Effects of the invention

[0022] According to various embodiments of the present disclosure, by maintaining the weights of the document encoder in a fixed state during the domain adaptation learning process of a document search model, the entire re-indexing process for a large-scale document database can be omitted, thereby preventing the consumption of computing resources and reducing infrastructure operating costs.

[0023] According to various embodiments of the present disclosure, by individually or integrally performing domain semantic learning through an adapter having a Low Rank Adaptation (LoRA) structure placed inside the transformer layer of a query encoder and geometric distribution matching learning through an adapter having a residual multilayer perceptron placed in a post-pooling stage, high search accuracy between a query and a document can be secured even in an asymmetric learning structure.

[0024] According to various embodiments of the present disclosure, by individually managing only lightweight adapter parameter sets for specific domains or merging multiple domain adapters using a weighted average method, search services for multiple domains can be flexibly expanded and dynamically loaded and provided in a single base model environment.

[0025] According to various embodiments of the present disclosure, by configuring the adapter to operate as an identity function at the initial stage of learning, destructive forgetting of general-purpose language knowledge already possessed by the base model can be prevented and the stability of transfer learning for domain adaptation can be maximized.

[0026] According to various embodiments of the present disclosure, by selectively combining adapters with various configurations of a query encoder for training, a high-performance document search model can be implemented with minimal parameter updates optimized for the characteristics of the target domain.

[0027] The effects of the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by a person skilled in the art to which the present disclosure pertains (referred to as "person skilled in the art") from the description in the claims. Brief explanation of the drawing

[0028] Embodiments of the present disclosure will be described with reference to the accompanying drawings described below, wherein similar reference numerals indicate similar elements, but are not limited thereto. FIG. 1 is a schematic diagram illustrating an example of a document search system according to one embodiment of the present disclosure. FIG. 2 is a schematic diagram showing a configuration in which an information processing system is connected to communicate with a plurality of user terminals in order to tune and use a document search model according to one embodiment of the present disclosure. FIG. 3 is a block diagram showing the internal configuration of a user terminal and an information processing system according to one embodiment of the present disclosure. FIG. 4 is a diagram showing an example of an internal hierarchy of a query encoder according to one embodiment of the present disclosure. FIG. 5 is a drawing showing an example of a first encoder according to one embodiment of the present disclosure. FIG. 6 is a drawing showing an example of a second encoder according to one embodiment of the present disclosure. FIG. 7 is a flowchart illustrating an example of a domain-adaptive learning method for a document search model according to one embodiment of the present disclosure. FIG. 8 is a flowchart illustrating an example of an adapter adaptive learning method for a document search model that can be integrated and applied to various domains according to one embodiment of the present disclosure. FIG. 9 is a flowchart illustrating an example of an information retrieval method using a query encoder comprising a first adapter and a second adapter, the parameters of which are optimized by a domain adaptive learning method of a document retrieval model according to one embodiment of the present disclosure. Embodiments of the present disclosure will be described with reference to the accompanying drawings described below, wherein similar reference numerals indicate similar elements, but are not limited thereto. Specific details for implementing the invention

[0029] Hereinafter, specific details for implementing the present disclosure will be described in detail with reference to the attached drawings. However, in the following description, specific descriptions regarding widely known functions or configurations will be omitted if there is a risk that the gist of the present disclosure may be unnecessarily obscured.

[0030] In the attached drawings, identical or corresponding components are assigned the same reference numerals. Additionally, in the description of the following embodiments, the description of identical or corresponding components may be omitted. However, even if a description of a component is omitted, it is not intended that such component is not included in any embodiment.

[0031] The advantages and features of the disclosed embodiments and the methods for achieving them will become clear by referring to the embodiments described below in conjunction with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below but may be implemented in various different forms, and the embodiments provided are merely to make the present disclosure complete and to fully inform those skilled in the art of the scope of the invention.

[0032] The terms used in this specification will be briefly explained, and the disclosed embodiments will be described in detail. The terms used in this specification have been selected to be as generally used as possible, taking into account their functions in this disclosure; however, these terms may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant description of the invention. Therefore, the terms used in this disclosure should be defined not merely by their names, but based on their meanings and the content throughout this disclosure.

[0033] In this specification, singular expressions include plural expressions unless the context clearly specifies them as singular. Additionally, plural expressions include singular expressions unless the context clearly specifies them as plural. Throughout the specification, when a part is described as including a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0034] Additionally, the terms 'module' or 'part' as used in the specification refer to software or hardware components, and the 'module' or 'part' performs certain roles. However, the meaning of 'module' or 'part' is not limited to software or hardware. The 'module' or 'part' may be configured to reside in an addressable storage medium or configured to run on one or more processors. Thus, as an example, the 'module' or 'part' may include components such as software components, object-oriented software components, class components, and task components, and at least one of processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, or variables. The components and the functions provided within the 'module' or 'part' may be combined into a smaller number of components and 'modules' or 'parts', or further separated into additional components and 'modules' or 'parts'.

[0035] According to one embodiment of the present disclosure, a ‘module’ or ‘part’ may be implemented as a processor and memory. The term ‘processor’ should be broadly interpreted to include a general-purpose processor, a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a controller, a microcontroller, a state machine, etc. In some environments, the term ‘processor’ may refer to an application-specific integrated circuit (ASIC), a programmable logic device (PLD), a field programmable gate array (FPGA), etc. The term ‘processor’ may also refer to a combination of processing devices, such as, for example, a combination of a DSP and a microprocessor, a combination of multiple microprocessors, a combination of one or more microprocessors combined with a DSP core, or any other combination of such configurations. Additionally, the term ‘memory’ should be broadly interpreted to include any electronic component capable of storing electronic information. 'Memory' may refer to various types of processor-readable media, such as Random Access Memory (RAM), Read-Only Memory (ROM), Non-Volatile Random Access Memory (NVRAM), Programmable Read-Only Memory (PROM), Erasable-Programmable Read-Only Memory (EPROM), Electrically Erasable PROM (EEPROM), Flash Memory, Magnetic or Optical Data Storage Devices, Registers, etc. If a processor can read information from memory and / or write information to memory, the memory is said to be in an electronic communication state with the processor. Memory integrated into a processor is in an electronic communication state with the processor.

[0036] In the present disclosure, the 'system' may include at least one of a server device and a cloud device, but is not limited thereto. For example, the system may be composed of one or more server devices. As another example, the system may be composed of one or more cloud devices. As yet another example, the system may be configured and operated with both a server device and a cloud device.

[0037] In the present disclosure, 'display' may refer to any display device associated with a computing device, for example, any display device capable of displaying any information / data controlled by or provided by the computing device.

[0038] In the present disclosure, 'each of a plurality of A' or 'each of a plurality of A' may refer to each of all components included in a plurality of A, or each of some components included in a plurality of A.

[0039] Various embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. The size or location of display screens, images, buttons, etc., as illustrated and described in the drawings are exemplary and are not limited thereto. For example, some buttons may be added or omitted, or configured with sizes and locations different from those illustrated. Furthermore, the flowcharts and descriptions illustrated in the drawings are merely examples and may be implemented differently in some embodiments. For example, one or more steps may be omitted, the order of each step may be changed, one or more steps may be performed in overlap, or one or more steps may be performed repeatedly.

[0040] FIG. 1 is a schematic diagram showing an example of a document search system (1) according to one embodiment of the present disclosure.

[0041] Referring to FIG. 1, the document search system (1) can provide search results optimized for a new domain while minimizing the cost of re-indexing a large document database. The document search system (1) may include a document encoder (10) and a query encoder (30) based on the weights of the base model of a dense retrieval model. That is, the document search system (1) may include a dual encoder structure.

[0042] The document encoder (10) can map documents into an embedding space. For example, the document encoder (10) can convert individual documents within a large document corpus into numerical vectors (document embeddings). The document encoder (10) retains the weights of the base model of the dense retrieval model, and the weights of the document encoder (10) can be fixed (frozen) so as not to change during the domain adaptive learning process. Accordingly, the cost and time of re-indexing, which would otherwise require re-encoding a large document database, can be saved.

[0043] The document embedding database (20) may be a space where pre-generated document embeddings are stored through the document encoder (10). The document embedding database (20) may form a document embedding space distribution.

[0044] The query encoder (30) can receive query text associated with a specific domain entered by the user and convert it into a vector-shaped query embedding. The query encoder (30) can be configured based on the weights of a dense retrieval model. The query vector generated by the query encoder (10) can be aligned to correspond to the document embedding spatial distribution of the document embedding database (20). To this end, an adapter (40) can be applied to the query encoder (30).

[0045] The adapter (40) may be a configuration added for domain adaptation while keeping the body weights of the query encoder (10) fixed. Search performance can be improved by learning a dataset of a specific domain through the adapter (40). The document search system (1) can tune a dense retrieval model using the adapter (40). Additionally, the document search system (1) may include an asymmetric tuning architecture that fixes the document-side embeddings and flexibly adjusts only the query-side embeddings.

[0046] The document search system (1) uses a tuned dense retrieval model to receive a user's query text associated with a specific domain, and vectorizes the query through a query encoder (10) to which an adapter (40) is applied, and in this process, the semantics of the domain are reflected and a result aligned with the document embedding space distribution can be produced.

[0047] The document search system (1) can calculate the similarity between the generated query embedding vector and the document vectors in the pre-built document embedding database (20). The document search system (1) can extract the optimal search result based on the calculated similarity and provide it to the user.

[0048] FIG. 2 is a schematic diagram showing a configuration in which an information processing system (230) is connected to communicate with a plurality of user terminals (210_1, 210_2, 210_3) in order to tune and use a document search model according to one embodiment of the present disclosure.

[0049] Referring to FIG. 2, the information processing system (230) may include system(s) capable of tuning and using a dense retrieval model. In one embodiment, the information processing system (230) may include a computer-executable program (e.g., a downloadable application) and one or more server devices and / or databases capable of storing, providing, and executing data related to a service for tuning a dense retrieval model and a service for using a tuned dense retrieval model, or one or more distributed computing devices and / or distributed databases based on a cloud computing service. For example, the information processing system (230) may include separate systems (e.g., servers) for a dense retrieval model tuning service and a dense retrieval model application service.

[0050] The dense retrieval model tuning service and dense retrieval model application service provided by the information processing system (230) can be provided to the user through applications installed on each of the plurality of user terminals (210_1, 210_2, 210_3).

[0051] Multiple user terminals (210_1, 210_2, 210_3) can communicate with an information processing system (230) through a network (220). The network (220) can be configured to enable communication between the multiple user terminals (210_1, 210_2, 210_3) and the information processing system (230). Depending on the installation environment, the network (220) may be configured as a wired network such as Ethernet, Power Line Communication, telephone line communication devices and RS-serial communication, a mobile communication network, a Wireless LAN (WLAN), Wi-Fi, Bluetooth and ZigBee, or a combination thereof. The communication method is not limited and may include not only communication methods utilizing communication networks that the network (220) may include (e.g., mobile communication network, wired internet, wireless internet, broadcasting network, satellite network, etc.) but also short-range wireless communication between user terminals (210_1, 210_2, 210_3).

[0052] In FIG. 2, a mobile phone terminal (210_1), a tablet terminal (210_2), and a PC terminal (210_3) are illustrated as examples of user terminals, but are not limited thereto, and the user terminals (210_1, 210_2, 210_3) may be any computing device capable of wired and / or wireless communication and capable of installing and running applications, etc. For example, user terminals may include smartphones, mobile phones, navigation systems, computers, laptops, digital broadcasting terminals, PDAs (Personal Digital Assistants), PMPs (Portable Multimedia Players), tablet PCs, game consoles, wearable devices, IoT (Internet of Things) devices, VR (Virtual Reality) devices, AR (Augmented Reality) devices, etc. Additionally, FIG. 2 illustrates three user terminals (210_1, 210_2, 210_3) communicating with an information processing system (230) through a network (220), but is not limited thereto, and may be configured so that a different number of user terminals communicate with an information processing system (230) through a network (220).

[0053] According to one embodiment, each of the user terminals (210_1, 210_2, 210_3) can receive information or data from another user terminal or transmit it to another user terminal through the network (220).

[0054] In FIG. 2, the information processing system (230) is shown as an independent device separated from the user terminals (210_1, 210_2, 210_3), but is not limited thereto, and the information processing system (230) may be implemented in an integrated manner with the user terminals (210_1, 210_2, 210_3).

[0055] FIG. 3 is a block diagram showing the internal configuration of a user terminal (210) and an information processing system (230) according to one embodiment of the present disclosure.

[0056] The user terminal (210) may refer to any computing device capable of running applications, web browsers, etc., and capable of wired / wireless communication, and may include, for example, the mobile phone terminal (210_1), tablet terminal (210_2), PC terminal (210_3) of FIG. 2. Referring to FIG. 3, the user terminal (210) may include a memory (312), a processor (314), a communication module (316), and an input / output interface (318). Similarly, the information processing system (230) may include a memory (332), a processor (334), a communication module (336), and an input / output interface (338). As illustrated in FIG. 3, the user terminal (210) and the information processing system (230) may be configured to communicate information and / or data through a network (220) using their respective communication modules (316, 336). Additionally, the input / output device (320) may be configured to input information and / or data to the user terminal (210) or output information and / or data generated from the user terminal (210) through the input / output interface (318).

[0057] The memory (312, 332) may include any non-transient computer-readable recording medium. According to one embodiment, the memory (312, 332) may include a permanent mass storage device such as ROM (read-only memory), a disk drive, a solid-state drive (SSD), or flash memory. As another example, a permanent mass storage device such as ROM, an SSD, flash memory, or a disk drive may be included in the user terminal (210) or information processing system (230) as a separate permanent storage device distinct from the memory. Additionally, an operating system and at least one program code may be stored in the memory (312, 332).

[0058] These software components may be loaded from a computer-readable recording medium separate from memory (312, 332). This separate computer-readable recording medium may include a recording medium that can be directly connected to the user terminal (210) and the information processing system (230), for example, a computer-readable recording medium such as a floppy drive, disk, tape, DVD / CD-ROM drive, or memory card. As another example, the software components may be loaded into memory (312, 332) via a communication module (316, 336) rather than a computer-readable recording medium. For example, at least one program may be loaded into memory (312, 332) based on a computer program installed by files provided through a network (220) by developers or a file distribution system that distributes installation files for the application.

[0059] The processor (314, 334) may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input / output operations. Instructions may be provided to the processor (314, 334) by memory (312, 332) or a communication module (316, 336). For example, the processor (314, 334) may be configured to execute instructions received according to program code stored in a recording device such as memory (312, 332).

[0060] The communication module (316, 336) may provide a configuration or function for the user terminal (210) and the information processing system (230) to communicate with each other via the network (220), and may provide a configuration or function for the user terminal (210) and / or the information processing system (230) to communicate with another user terminal or another system (e.g., a separate cloud system). For example, a request or data generated by the processor (314) of the user terminal (210) according to program code stored in a recording device such as memory (312) may be transmitted to the information processing system (230) via the network (220) under the control of the communication module (316). Conversely, a control signal or command provided under the control of the processor (334) of the information processing system (230) may be received by the user terminal (210) via the communication module (316) of the user terminal (210) through the communication module (336) and the network (220).

[0061] The input / output interface (318) may be a means for interfacing with an input / output device (320). As an example, the input device may include a device such as a camera including an audio sensor and / or an image sensor, a keyboard, a microphone, or a mouse, and the output device may include a device such as a display, a speaker, or a haptic feedback device. As another example, the input / output interface (318) may be a means for interfacing with a device in which the configuration or function for performing input and output is integrated into one, such as a touchscreen. For example, when the processor (314) of the user terminal (210) processes instructions of a computer program loaded in memory (312), a service screen configured using information and / or data provided by an information processing system (230) or another user terminal may be displayed on a display through the input / output interface (318). In FIG. 3, the input / output device (320) is depicted as not being included in the user terminal (210), but is not limited thereto and may be configured as a single device with the user terminal (210). Additionally, the input / output interface (338) of the information processing system (230) may be a means for interfacing with a device (not shown) for input or output that is connected to the information processing system (230) or that the information processing system (230) may include. In FIG. 3, the input / output interface (318, 338) is shown as an element configured separately from the processor (314, 334), but is not limited thereto, and the input / output interface (318, 338) may be configured to be included in the processor (314, 334).

[0062] The user terminal (210) and the information processing system (230) may include more components than those of FIG. 3. However, it is not necessary to clearly illustrate most of the prior art components. In one embodiment, the user terminal (210) may be implemented to include at least some of the input / output devices (320) described above. Additionally, the user terminal (210) may further include other components such as a transceiver, a GPS (Global Positioning System) module, a camera, various sensors, a database, etc. For example, if the user terminal (210) is a smartphone, it may include components that are generally included in a smartphone, and may be implemented to include various components such as an accelerometer, a gyroscope, a microphone module, a camera module, various physical buttons, buttons using a touch panel, input / output ports, and a vibrator for vibration.

[0063] While a program for an application including a dense retrieval model tuning service and / or a dense retrieval model application service is running, the processor (314) can receive text, images, video, voice and / or actions, etc. that are input or selected through an input device such as a touch screen, keyboard, audio sensor and / or image sensor, camera, microphone, etc. connected to an input / output interface (318), and can store the received text, images, video, voice and / or actions, etc. in memory (312) or provide them to an information processing system (230) through a communication module (316) and a network (220).

[0064] The processor (314) of the user terminal (210) may be configured to manage, process, and / or store information and / or data received from an input / output device (320), another user terminal, an information processing system (230), and / or a plurality of external systems. The information and / or data processed by the processor (314) may be provided to the information processing system (230) through a communication module (316) and a network (220). The processor (314) of the user terminal (210) may transmit information and / or data to the input / output device (320) through an input / output interface (318) and output it. For example, the processor (314) may display the received information and / or data on the screen of the user terminal (210).

[0065] The processor (334) of the information processing system (230) may be configured to manage, process, and / or store information and / or data received from a plurality of user terminals (210) and / or a plurality of external systems. The information and / or data processed by the processor (334) may be provided to the user terminals (210) through a communication module (336) and a network (220).

[0066] In the following, the processor may include the processor of the information processing system (230) or the processor of the user terminal (210).

[0067] FIG. 4 is a diagram illustrating an example of the internal hierarchical structure of a query encoder according to one embodiment of the present disclosure. FIG. 5 is a diagram illustrating an example of a first encoder according to one embodiment of the present disclosure. FIG. 6 is a diagram illustrating an example of a second encoder according to one embodiment of the present disclosure. FIG. 5 and FIG. 6 will be referenced together where necessary while describing FIG. 4. The query encoder (400) may be initially configured based on the weights of the base model of the document search model.

[0068] Referring to FIG. 4, the processor can receive a query text. The processor can vectorize the query text through an input embedding layer (410). To prevent the location information from being lost, the processor can add a unique vector representing absolute or relative location information through a location encoding (420) and provide it to the transformer block (430).

[0069] The processor may include a plurality of transformer blocks (430) in a stacked structure while configuring the query encoder (400). The processor can identify basic relationships or grammatical structures between words through blocks (lower blocks) close to the input embedding layer (410), and can understand the complex meaning of a sentence or domain-specific context more deeply through blocks (upper blocks) close to the output vector (450). The processor may configure the transformer blocks (430) to be 12 or 24, but is not limited thereto.

[0070] The transformer block (430) may include a multi-head attention layer (432), an add-and-normal layer (434), a feed-forward network layer (436), and an add-and-normal layer (438).

[0071] First, the multi-head attention layer (432) can analyze the correlation between words within the query text to focus on contextually important information. The residual connection and normalization layer (434) can perform residual connection by directly adding the input value of the multi-head attention layer to the output value, and then perform layer normalization. Through the residual connection and normalization layer (434), information loss that may occur during the process of identifying the correlation between words within the query text can be prevented.

[0072] Additionally, the feedforward neural network layer (436) can generate higher-dimensional information by non-linearly transforming the features extracted through the attention process at each position. The residual linkage and normalization layer (438) can perform normalization once again after adding the input value to the output value of the feedforward neural network layer (436). If a transformer block is placed after the transformer block (430), the output of the residual linkage and normalization layer (438) can be passed to the transformer block placed next. If the transformer block (430) is the last one, an output vector (450) can be output.

[0073] The first encoder (440) can be placed in the transformer layer of the query encoder (400) for semantic learning. The first encoder (440) can extract domain-specific semantics while freezing the body weights of the base model. The first encoder (440) can be trained by being coupled to at least one of the following: an embedding matrix associated with the input embedding layer (410) of the query encoder (440), a projection matrix associated with the query, key, value, and output included in the multi-head attention layer (432) of the transformer layer, or a linear layer included in the feed-forward neural network layer (436) of the transformer layer.

[0074] Referring together with FIG. 5, the first encoder (500) (440) receives a d-dimensional input vector (x, 510) and can perform a first operation (520) and a second operation (530) based on the original weights (W) of the base model. The first encoder (500) (440) can finally output a result value (h) (540) through an addition operation on the result values ​​of the first operation (520) and the second operation (530). Specifically, the result value (h) can be calculated based on [Equation 1] below. W is the original weight of the base model and may include a matrix of size d rows and d columns in the range of real numbers (R). d may be a dimension associated with the base model. A may be a first matrix (532) placed in parallel with the weights (W) of the base model. The first matrix (532) can be initialized with a Gaussian distribution (N) and can perform a Projection Down operation that receives an input vector (510) and outputs a rank (r1)-based value. In the case of B, it may be a second matrix (534) placed in parallel with the weight (W) of the base model. The second matrix (534) can be initialized with a zero matrix and configured to preserve the output of the base model during the initial training. The second matrix (534) can perform a Projection Up operation that receives a rank (r1)-based value and outputs a d-dimensional value.

[0075] Referring to FIG. 5, the structure of the first adapter is described in detail. The first adapter may include a first matrix (A, 532) having dxr dimensions and a second matrix (B, 534) having rxd dimensions. Here, d represents the hidden dimension of the base model, and r1 may represent the rank of the adapter.

[0076] According to one embodiment of the present disclosure, the rank r1 is set to a significantly smaller integer than d (r1 << d, e.g., 4, 8, 16, etc.), thereby allowing the number of trainable parameters to be reduced to 0.1% to 10% of the total parameters of the base model. This can provide the effect of preventing catastrophic forgetting while significantly improving the learning speed.

[0077] [Formula 1]

[0078] h = Wx + BAx,

[0079] Referring again to FIG. 4, a second adapter (460) for distribution alignment may be placed at the output end after pooling of the query encoder (400). The second adapter (460) may operate as an identity function that outputs an input vector without modification at the start of learning with the dataset of the first domain. The second adapter (460) may be trained to geometrically align the query embedding vector (470) generated through the pooling of the query encoder (400) with the document embedding space distribution already constructed by the document encoder.

[0080] Referring together with FIG. 6, and referring to the configuration (600) after the transformer block (430), the second adapter (620) (460) may include a residual linkage structure that adds a non-linear correction value calculated through a multilayer perceptron to the output vector (610) (450) after pooling. The query encoder (400) may finally output a query embedding vector (630) (470) based on the output vector (610) (450) and the second adapter (620) (460).

[0081] Here, the multilayer perceptron may be composed of multiple layers containing nonlinear activation functions. The multiplication structure of the low-rank-based first matrix (A) and the low-rank-based second matrix (B) can be visualized as the linear layers of the multilayer perceptron forming a bottleneck structure through the rank (r2). The rank (r2) can be set independently of the rank (r1) shown in FIG. 5. For example, if the difficulty of spatial distribution matching is higher than that of domain semantic learning, the rank r2 can be set higher than the rank r1 to increase the precision of the correction, but is not limited thereto.

[0082] A non-linear activation function may be included between the first matrix (A) and the second matrix (B) of the multilayer perceptron. The non-linear activation function may be either ReLU (Rectified Linear Unit) or GeLU (Gaussian Error Linear Unit), thereby enabling the second adapter (620) (460) to produce complex non-linear correction values ​​beyond simple linear transformations for the output vector (610) (450).

[0083] According to one embodiment, the query embedding vector (630)(470) can be calculated based on [Equation 2] below. Here, Qf is the query embedding vector (630)(470), Vo is the output vector (610)(450), and MLP(Vo) is a non-linear correction value calculated by the second adapter (620)(460). This residual linkage structure can induce the query encoder (400) to perform fine positioning into the document embedding space while maintaining the original semantic information (Vo) learned as much as possible.

[0084] [Equation 2]

[0085] Qf = Vo + MLP(Vo),

[0086] According to one embodiment, in order to ensure the stability of the second adapter (620) during the initial learning phase, the present disclosure may adopt a specific initialization strategy. Specifically, in the residual connection structure (Qf = Vo + MLP(Vo)), the weight of the last layer of the MLP may be initialized to '0 (Zero)'. Accordingly, at the start of learning (Step=0), the output value of the MLP(Vo) becomes a 0 vector, and the final output Qf may be the same as the input Vo (Qf = Vo). This may be a technical device that prevents the query encoder from destroying previously learned linguistic knowledge due to random noise during the initial learning phase and induces it to gradually align the distribution into the document embedding space.

[0087] According to one embodiment, the second adapter (620) (460) can perform scaling, rotation, or translation operations on the vectors to match the query embedding vectors (630) (470) to the statistical distribution (e.g., mean and variance) of the pre-established document embedding space. This allows the efficiency of calculations for the cosine similarity or inner product between the query and the document to be improved by relatively shifting only the query vectors while the document encoder is frozen and the positions of the document vectors are fixed.

[0088] Geometric alignment performed by the second adapter above may refer to the process of positioning query vectors on a manifold formed by frozen document embedding vectors by rotating, translating, or scaling the query vector space. To this end, during the learning process, an in-batch negative-based InfoNCE loss function or a triplet loss function may be used to maximize the dot product similarity between positive passages associated with the query and minimize the similarity between negative passages.

[0089] FIG. 7 is a flowchart illustrating an example of a domain-adaptive learning method for a document search model according to one embodiment of the present disclosure.

[0090] In step S710, the processor can configure a query encoder and a document encoder based on the weights of the base model. Here, the weights of the document encoder can be frozen so that they do not change during the training process.

[0091] According to one embodiment of the present disclosure, the base model may include a pre-trained language model such as BERT, RoBERTa, DistilBERT, or ELECTRA based on a transformer architecture. In this case, the query encoder and the document encoder may be initialized with a Siamese network structure that shares the same weights, or with a structure that has different weights. Additionally, the transformer layer of the query encoder mentioned in this specification may refer to a block that includes at least one multi-head attention structure among a plurality of layers constituting the query encoder.

[0092] In addition, pooling refers to the process of extracting a vector representing the meaning of the entire sentence from the last hidden state output of the transformer layer, and this may include at least one of the following methods: using the vector of the [CLS] token, taking the average (Mean Pooling) of all token vectors, or taking the maximum value (Max Pooling).

[0093] In step S720, the processor may place a first adapter for semantic learning in the transformer layer of the query encoder and a second adapter for distribution alignment in the output after pooling of the query encoder.

[0094] In step S730, the processor can update the parameters of the first adapter and the second adapter while fixing the body weights of the query encoder using the dataset of the first domain. Here, the second adapter can operate as an identity function that outputs the input vector without modification at the start of learning using the dataset of the first domain.

[0095] According to one embodiment, the first adapter may be trained to reflect the domain semantics of text included in the dataset of the first domain. The second adapter may be trained to geometrically align the query embedding vector generated through pooling of the query encoder with the document embedding space distribution already constructed by the document encoder.

[0096] According to one embodiment, a document embedding database generated in advance through a document encoder can correspond to a document embedding spatial distribution, and an update process or a re-indexing process may be omitted while the domain adaptive learning method is being performed or after it has been completed.

[0097] According to one embodiment, the second adapter may include a residual linkage structure that adds a non-linear correction value calculated through a multilayer perceptron to the output vector after pooling.

[0098] According to one embodiment, a multilayer perceptron is composed of a plurality of layers including a nonlinear activation function, wherein the weights of the last layer of the multilayer perceptron may be initialized to a zero matrix. The nonlinear activation function included in the multilayer perceptron may include either ReLU (Rectified Linear Unit) or GeLU (Gaussian Error Linear Unit), but is not limited thereto.

[0099] According to one embodiment, the first adapter can be trained by being coupled to at least one of the following: an embedding matrix associated with the input embedding layer of the query encoder, a projection matrix associated with the query, key, value, and output included in the multi-head attention layer of the transformer layer, or a linear layer included in the feed-forward network layer of the transformer layer.

[0100] According to one embodiment, the first adapter is composed of the product of a low-rank based first matrix and a second matrix arranged in parallel with the original weight matrix of the base model, wherein the first matrix is ​​initialized with a Gaussian distribution and the second matrix is ​​initialized with a zero matrix so as to preserve the output of the base model during the initial training.

[0101] FIG. 8 is a flowchart illustrating an example of an adapter adaptive learning method for a document search model that can be integrated and applied to various domains according to one embodiment of the present disclosure.

[0102] In step S810, the processor can update the parameters of a plurality of third adapters associated with a first adapter while fixing the body weights of the query encoder with different domain datasets.

[0103] In step S820, the processor can generate an integrated adapter adapted to multiple domains by merging the parameters of multiple updated third adapters through a weighted average method.

[0104] According to one embodiment, the weighted average method performed in step S820 may include not only an arithmetic average method that assigns equal weight to the parameter values ​​of each adapter, but also a weighted arithmetic average method that assigns different weights based on the size or importance of each domain dataset and sums them. Through this, it is possible to control the influence of a specific domain or generate an integrated adapter that optimizes the balance between general-purpose search performance and domain-specific performance.

[0105] According to one embodiment, by merging multiple third adapters individually trained using datasets specialized for each domain (e.g., law, medical, finance, etc.), an integrated search model that encompasses a wide range of expertise, not limited to a single domain, can be constructed. This significantly reduces training time compared to a method of retraining a massive model by combining all domain data from the beginning, and provides scalability that can flexibly respond whenever a new domain is added.

[0106] In step S830, the processor can replace the first adapter with the generated integrated adapter.

[0107] According to one embodiment, the processor can use a dataset of the second domain to obtain parameter sets of the updated fourth adapter and fifth adapter while fixing the body weights of the query encoder, and dynamically load the parameter sets of the fourth adapter and fifth adapter into the base model.

[0108] According to one embodiment, the process of replacing the first adapter with the integrated adapter generated in step S830 can be performed by replacing only the values ​​in the adapter parameter memory area while maintaining the existing structure of the query encoder. Through this, the query encoder acquires general-purpose domain adaptability capable of effectively processing multi-domain queries with only a single inference process, without configuring a separate additional computation layer.

[0109] FIG. 9 is a flowchart illustrating an example of an information retrieval method using a query encoder comprising a first adapter and a second adapter, the parameters of which are optimized by a domain adaptive learning method of a document retrieval model according to one embodiment of the present disclosure.

[0110] In step S910, the processor can receive query text associated with the first domain.

[0111] In step S920, the processor can tokenize the query text and generate a query vector associated with the query text through the input embedding layer of the query encoder.

[0112] In step S930, the processor can generate a query embedding vector based on the query vector. The query embedding vector can reflect the domain semantics of the first domain through the first adapter and be aligned on the document embedding space distribution through the second adapter.

[0113] According to one embodiment, the processor can identify a domain from the received query text and dynamically load a pre-learned set of adapter parameters corresponding to the identified domain into the query encoder.

[0114] Dynamic loading according to one embodiment of the present disclosure may refer to a process of residing a large weight of a base model (e.g., hundreds of MB to several GB) in the memory of an information processing system, and, as a user request domain (e.g., law, medical) is identified, loading only a lightweight adapter parameter file (e.g., several MB) corresponding to that domain from storage in real time to update the value of the adapter area in memory. Through this, domain switching in milliseconds (ms) may be possible within a single process without restarting a separate model.

[0115] According to one embodiment, when a processor receives a search request for a second domain different from a first domain, it can perform a search by replacing the adapter parameter set of a query encoder with a parameter set updated to correspond to the second domain without a re-indexing process of the document embedding database.

[0116] In step S940, the processor can calculate the similarity between the query embedding vectors and the document vectors in the document embedding database pre-built by the document encoder with frozen weights.

[0117] According to one embodiment, the processor can perform a cosine similarity or inner product operation between the generated query embedding vectors and document vectors.

[0118] In step S950, the processor can extract and provide search results based on the calculated similarity.

[0119] According to one embodiment, the processor can extract a preset number of documents in order of highest calculated similarity and provide the extracted documents sorted in a list form.

[0120] The method described above may be provided as a computer program stored on a computer-readable recording medium for execution on a computer. The medium may continuously store a program executable by a computer, or temporarily store it for execution or download. Additionally, the medium may be various recording or storage means in the form of a single or multiple hardware components combined, and may not be limited to a medium directly connected to a computer system but may exist distributed over a network. Examples of media may include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and media configured to store program instructions, including ROM, RAM, and flash memory. Furthermore, other examples of media may include recording or storage media managed by app stores that distribute applications or sites and servers that supply or distribute various other software.

[0121] The methods, operations, or techniques of the present disclosure may be implemented by various means. For example, these techniques may be implemented in hardware, firmware, software, or a combination thereof. Those skilled in the art will understand that the various exemplary logical blocks, modules, circuits, and algorithmic steps described in connection with the disclosure herein may be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate such interchangeability between hardware and software, various exemplary components, blocks, modules, circuits, and steps have been generally described above in terms of their functional aspects. Whether such functions are implemented in hardware or in software depends on the design requirements imposed on the specific application and the overall system. Those skilled in the art may implement the functions described in various ways for each specific application, but such implementations should not be construed as departing from the scope of the present disclosure.

[0122] In a hardware implementation, the processing units used to perform the techniques may be implemented in one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, other electronic units designed to perform the functions described in this disclosure, computers, or a combination thereof.

[0123] Accordingly, the various exemplary logic blocks, modules, and circuits described in connection with the present disclosure may be implemented or performed by any combination of general-purpose processors, DSPs, ASICs, FPGAs or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or those designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, for example, a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors coupled with a DSP core, or any other combination of configurations.

[0124] In firmware and / or software implementations, techniques may be implemented as instructions stored on a computer-readable medium such as random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, compact disc (CD), magnetic or optical data storage devices, etc. The instructions may be executable by one or more processors, and the processor(s) may be enabled to perform specific aspects of the functions described in this disclosure.

[0125] Where implemented in software, techniques may be stored on a computer-readable medium as one or more instructions or code, or transmitted through a computer-readable medium. Computer-readable media include both computer storage media and communication media, including any medium that facilitates the transmission of a computer program from one place to another. Storage media may be any available medium accessible by a computer. As a non-limiting example, such computer-readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium accessible by a computer that can be used to transfer or store desired program code in the form of instructions or data structures. Additionally, any connection is appropriately referred to as a computer-readable medium.

[0126] For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair cable, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, coaxial cable, fiber optic cable, twisted pair cable, digital subscriber line, or wireless technologies such as infrared, radio, and microwave are included within the definition of a medium. As used herein, disk and disc include CD, laser disc, optical disc, DVD (digital versatile disc), floppy disk, and Blu-ray disc, wherein disks usually play data magnetically, whereas discs play data optically using a laser. The above combinations should also be included within the scope of computer-readable media.

[0127] The software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, a hard disk, a removable disk, a CD-ROM, or any other known form of storage medium. An exemplary storage medium may be connected to a processor so that the processor can read information from the storage medium or write information to the storage medium. Alternatively, the storage medium may be integrated into the processor. The processor and the storage medium may exist within an ASIC. The ASIC may exist within a user terminal. Alternatively, the processor and the storage medium may exist as separate components within the user terminal.

[0128] Although the embodiments described above have been described as utilizing aspects of the subject matter disclosed herein in one or more standalone computer systems, the present disclosure is not limited thereto and may be implemented in conjunction with any computing environment, such as a network or a distributed computing environment. Furthermore, aspects of the subject matter in the present disclosure may be implemented in a plurality of processing chips or devices, and storage may be similarly affected across a plurality of devices. Such devices may include PCs, network servers, and portable devices.

[0129] Although the present disclosure has been described in relation to some embodiments, various modifications and changes may be made without departing from the scope of the present disclosure as understood by a person skilled in the art to which the invention of the present disclosure pertains. Furthermore, such modifications and changes should be considered to fall within the scope of the claims appended to this specification. Explanation of the symbols

[0130] 1: Document Search System 10: Document Encoder 20: Document Embedding Database 30: Query Encoder 40: Adapter

Claims

Claim 1 A domain adaptive learning method for a document search model, performed by at least one processor, comprising the steps of: configuring a query encoder and a document encoder based on the weights of a base model — the weights of the document encoder are frozen so as not to be changed during the learning process —; placing a first adapter for semantic learning in the transformer layer of the query encoder and placing a second adapter for distribution alignment in the output after pooling of the query encoder; A domain adaptive learning method comprising the step of updating the parameters of the first adapter and the second adapter while fixing the body weights of the query encoder using a dataset of the first domain, wherein the second adapter is configured such that the last layer weights of the multilayer perceptron included in the second adapter are initialized to a zero matrix, and at the start of learning using the dataset of the first domain, it operates as an identity function that outputs an input vector without modification, and acts on a single query embedding vector representing the meaning of the entire input text as a pooling result for a plurality of token vectors included in the last hidden state of the transformer layer, and as learning progresses, by adjusting the vector space arrangement relationship of the single query embedding vector with respect to the document embedding space distribution formed by document embedding vectors that are pre-generated by the document encoder with frozen weights and stored in the document embedding database, a geometric alignment relationship is gradually formed between the single query embedding vector and the document embedding space distribution. Claim 2 A domain adaptive learning method according to claim 1, wherein the first adapter is trained to reflect the domain semantics of text included in the dataset of the first domain, and the second adapter is trained to form a geometric alignment relationship between the single query embedding vector and the document embedding spatial distribution constructed by the document encoder by performing at least one of scaling, rotation, or translation operations on the single query embedding vector generated through pooling of the query encoder. Claim 3 A domain adaptive learning method according to claim 1, wherein the second adapter includes a residual linkage structure that adds a non-linear correction value calculated through a multilayer perceptron to the output vector after pooling. Claim 4 In paragraph 3, the multilayer perceptron is a domain adaptive learning method composed of a plurality of layers including a non-linear activation function. Claim 5 A domain adaptive learning method according to claim 1, wherein the first adapter is coupled and learned at least one of an embedding matrix associated with an input embedding layer of the query encoder, a projection matrix associated with a query, key, value, and output included in a multi-head attention layer of the transformer layer, or a linear layer included in a feed-forward network layer of the transformer layer. Claim 6 A domain adaptive learning method according to claim 1, wherein the first adapter is composed of the product of a low-rank based first matrix and a second matrix arranged in parallel with the original weight matrix of the base model, wherein the first matrix is ​​initialized with a Gaussian distribution and the second matrix is ​​initialized with a zero matrix so as to preserve the output of the base model during the initial learning phase. Claim 7 In claim 1, the document embedding database generated in advance through the document encoder is a domain adaptive learning method in which an update process or a re-indexing process is omitted while the domain adaptive learning method is being performed or after it has been completed. Claim 8 A domain adaptive learning method according to claim 1, further comprising: a step of updating parameters of a plurality of third adapters associated with a first adapter while fixing the body weights of the query encoder with different domain datasets; a step of merging the updated parameters of the plurality of third adapters through a weighted average method to generate an integrated adapter adapted to a plurality of domains; and a step of replacing the first adapter with the generated integrated adapter. Claim 9 A domain adaptive learning method according to claim 1, further comprising: a step of obtaining parameter sets of an updated fourth adapter and a fifth adapter while fixing the body weights of the query encoder using a dataset of the second domain; and a step of dynamically loading the parameter sets of the fourth adapter and the fifth adapter into the base model. Claim 10 In claim 4, the non-linear activation function included in the multilayer perceptron comprises either ReLU (Rectified Linear Unit) or GeLU (Gaussian Error Linear Unit), a domain adaptive learning method. Claim 11 A method for retrieving information using a query encoder comprising a first adapter and a second adapter, the parameters of which are optimized by a domain adaptive learning method, comprising: receiving a query text associated with a first domain; tokenizing the query text and generating a query vector associated with the query text through an input embedding layer of the query encoder; generating a query embedding vector based on the query vector — wherein the query embedding vector reflects the domain semantics of the first domain through the first adapter and is aligned on a document embedding space distribution through the second adapter —; and calculating a similarity between the query embedding vector and document vectors in a document embedding database pre-constructed by a document encoder with frozen weights. The information retrieval method includes the step of extracting and providing search results based on the similarity calculated above, wherein the second adapter is placed at the output end after pooling of the query encoder and acts on a single query embedding vector representing the meaning of the entire query text as a pooling result for a plurality of token vectors included in the last hidden state of the transformer layer, wherein the weight of the last layer of the multilayer perceptron included in the second adapter is initialized to a zero matrix at the time of starting learning by the dataset of the first domain in the domain adaptive learning method, and the second adapter operates as an identity function that outputs the input vector without modification, and wherein, as learning proceeds in the domain adaptive learning method, the second adapter is learned to gradually form a geometric alignment relationship between the single query embedding vector and the document embedding spatial distribution by adjusting the vector spatial arrangement relationship of the single query embedding vector with respect to the document embedding spatial distribution formed by document embedding vectors that are pre-generated by the document encoder with frozen weights and stored in the document embedding database. Claim 12 In claim 11, the step of generating the query embedding vector comprises: identifying a domain from the received query text; and dynamically loading a pre-learned set of adapter parameters corresponding to the identified domain into the query encoder, in an information retrieval method. Claim 13 A method for retrieving information according to claim 11, further comprising: receiving a search request for a second domain different from the first domain; and performing a search by replacing the adapter parameter set of the query encoder with an updated parameter set corresponding to the second domain without a re-indexing process of the document embedding database. Claim 14 In claim 11, the step of calculating the similarity includes a step of performing a cosine similarity or dot product operation between the generated query embedding vector and the document vectors, and the step of providing the search results includes a step of extracting a preset number of documents in order of highest to lowest similarity, and providing the extracted documents sorted in a list form. Claim 15 In an information processing system, memory; and includes a processor connected to the memory and configured to execute at least one computer-readable program contained in the memory, wherein the at least one program comprises a query encoder and a document encoder based on weights of a base model, a first adapter for semantic learning placed in the transformer layer of the query encoder, a second adapter for distribution alignment placed at the output end after pooling of the query encoder, and instructions for updating the parameters of the first adapter and the second adapter while fixing the body weights of the query encoder using a dataset of the first domain, wherein the weights of the document encoder are frozen so as not to be changed during the learning process, and the second adapter is configured to operate as an identity function in which the last layer weights of the multilayer perceptron included in the second adapter are initialized to a zero matrix, and at the start of learning using the dataset of the first domain, the input vector of the second adapter is output without modification, and a single query representing the semantics of the entire input text as a pooling result for a plurality of token vectors included in the last hidden state of the transformer layer An information processing system configured to gradually form a geometric alignment relationship between the single query embedding vector and the document embedding spatial distribution by acting on the embedding vector and, as learning progresses, adjusting the vector spatial arrangement relationship of the single query embedding vector with respect to the document embedding spatial distribution formed by the document embedding vectors that are pre-generated by the document encoder with frozen weights and stored in the document embedding database.

Images