A method and apparatus for accessing a heterogeneous search augmented generation (RAG) platform

Abstract

The application discloses an access method and device of a heterogeneous retrieval enhancement generation RAG platform, comprising the following steps: sending target routing information and original input parameter information to a platform request composer; converting the original input parameter information into target input parameter information matched with a to-be-accessed heterogeneous RAG platform through the platform request composer, and sending the target input parameter information to the to-be-accessed heterogeneous RAG platform through the target routing information; determining an explicit response and an implicit response based on a logical mark in response information, and extracting the explicit response to feed back to a standard application. The request information is converted into request information matched with the to-be-accessed heterogeneous RAG platform through an OpenAI protocol conversion gateway, without the need to separately develop codes for each heterogeneous RAG platform, and the responses fed back by each heterogeneous platform are converted into explicit and implicit responses according to the format of a marked application, which can be displayed in the fields of a standard application, thereby improving the access effect of each heterogeneous RAG platform and the consistency and stability of an upper-layer application.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of information processing, and in particular to an access method and device of a heterogeneous retrieval augmented generation RAG platform. BACKGROUND

[0002] In a large language model retrieval augmented generation (RAG) system, the Chat Completions interface provided by OpenAI has become a general standard interface of the large language model application ecosystem. At present, mainstream RAG platforms, such as the RAGFlow platform or the Dify platform, although follow the general technical path of “vector retrieval, context splicing and large model generation”, there are still significant differences in parameter naming, event semantics and streaming end markers compared with the standard application OpenAI.

[0003] At present, if a business application needs to access the above-mentioned heterogeneous RAG platforms, it must be customized to develop adaptation code for each RAG platform, which has high development and maintenance costs. Due to the different event naming and end markers used by each RAG platform in streaming transmission, the application layer logic is complex and inconsistent in behavior, and the reasoning content of each platform is inconsistent, which makes it difficult for the application system to stably convert to the standard application field for display, thereby affecting the access effect of each heterogeneous RAG platform. SUMMARY

[0004] The present application provides an access method and device of a heterogeneous retrieval augmented generation RAG platform to realize the access to the heterogeneous RAG platform.

[0005] According to a first aspect of the present application, an access method of a heterogeneous retrieval augmented generation RAG platform is provided, which is applied to an OpenAI protocol conversion gateway, and the method comprises:

[0006] Obtaining request information of a current session, wherein the request information comprises original input parameter information and standard application routing information;

[0007] Converting the standard application routing information into target routing information matched with a heterogeneous RAG platform to be accessed, and sending the target routing information and the original input parameter information to a platform request composer;

[0008] Converting the original input parameter information into target input parameter information matched with the heterogeneous RAG platform to be accessed by the platform request composer, and sending the target input parameter information to the heterogeneous RAG platform to be accessed through the target routing information;

[0009] obtain response information generated by the heterogeneous RAG platform for the target input parameter information, determine explicit response and implicit response based on logical markers in the response information, and extract the explicit response to feed back to the standard application.

[0010] According to another aspect of the present application, an access device of a heterogeneous retrieval enhancement generation RAG platform is provided, comprising: a request information obtaining module configured to obtain request information of a current session, the request information including original input parameter information and standard application routing information;

[0011] a request information routing module configured to convert the standard application routing information into target routing information matched with a heterogeneous RAG platform to be accessed, and send the target routing information and the original input parameter information to a platform request orchestrator;

[0012] an original input parameter information conversion module configured to convert the original input parameter information into target input parameter information matched with the heterogeneous RAG platform to be accessed by the platform request orchestrator, and send the target input parameter information to the heterogeneous RAG platform to be accessed through the target routing information;

[0013] a response feedback module configured to obtain response information generated by the heterogeneous RAG platform for the target input parameter information, determine explicit response and implicit response based on logical markers in the response information, and extract the explicit response to feed back to the standard application.

[0014] According to another aspect of the present application, a terminal device is provided, comprising: one or more processors;

[0015] a storage device configured to store one or more programs,

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any embodiment of the present application.

[0017] According to another aspect of the present application, a storage medium of computer executable instructions is provided, and a computer program is stored on the storage medium, which is executed by a processor to implement the method described in any embodiment of the present application.

[0018] The technical scheme of the present application, through unified open standard application, obtains the request information of the current session, and converts the request information into the request information matched with the to-be-accessed heterogeneous RAG platform through the OpenAI protocol conversion gateway, so as to realize the access to each heterogeneous RAG platform without separately developing the code for each heterogeneous RAG platform, thereby reducing the development cost, and according to the response feedback of each heterogeneous platform, the explicit and implicit conversion is performed according to the format of the marked application, so that the standard application field can be displayed, the access effect of each heterogeneous RAG platform and the consistency and stability of the upper application are improved.

[0019] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present application, nor to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0021] Figure 1 is a flow chart of an access method of a heterogeneous search enhanced generation RAG platform according to the first embodiment of the present application;

[0022] Figure 2 is a structural schematic diagram of an OpenAI protocol conversion gateway according to the first embodiment of the present application;

[0023] Figure 3 is a flow chart of an access method of a heterogeneous search enhanced generation RAG platform according to the second embodiment of the present application;

[0024] Figure 4 is a streaming request timing diagram according to the second embodiment of the present application;

[0025] Figure 5 is a structural schematic diagram of an access device of a heterogeneous search enhanced generation RAG platform according to the second embodiment of the present application;

[0026] Figure 6 is a structural block diagram of a terminal device according to the fourth embodiment of the present application. DETAILED DESCRIPTION

[0027] In the following, the technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application in order to make the technical personnel in the technical field better understand the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all the other embodiments obtained by the ordinary technical personnel in the technical field without creative work should belong to the protection scope of the present application.

[0028] It should be noted that the terms "first", "second" and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, device, product or terminal device including a series of steps or units need not be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or terminal devices.

[0029] Embodiment one

[0030] Figure 1 A flowchart of an access method of a heterogeneous retrieval enhancement generation RAG platform provided by the embodiment of the present application, the embodiment can be applicable to the case of accessing the heterogeneous RAG platform, the method can be executed by an access device of a heterogeneous retrieval enhancement generation RAG platform, the device can be realized in the form of hardware and / or software, and the device can be integrated in a terminal device. As shown in the figure, the method comprises: Figure 1

[0031] Step S101, obtaining request information of a current session, the request information comprising original input parameter information and standard application routing information.

[0032] In the embodiment, the method comprises: Figure 2 ​As shown is a structural schematic diagram of the OpenAI protocol conversion gateway. Through the OpenAI protocol conversion gateway, the request information uniformly received by the standard application interface can be converted into a format adapted to the heterogeneous RAG platform to be accessed, and the response feedback by the heterogeneous RAG platform can be converted into a standard application compatible protocol format. The core solves key problems such as Server-Sent Events (SSE) normalization, response content differential increment, error handling shunting, reasoning content mapping and explicit / implicit control, and session unified management. In the OpenAI protocol conversion gateway, it specifically includes an entry protocol adapter, a security and operation and maintenance component, a configuration center, a platform request orchestrator, an SSE normalization and differentiator, a reasoning explicit / implicit extractor, and a session context manager. For the SSE normalization and differentiator, it is only involved when the request information sent to the heterogeneous platform is a streaming request. For non-streaming requests, the component is not involved in the processing process. Of course, in the present embodiment, only an example is given, and the specific structure of the OpenAI protocol conversion gateway is not limited. Only when the access to each heterogeneous RAG platform can be realized through the standard application without additional code development, it is within the protection scope of the present application.

[0033] Optionally, the request information of the current session is obtained, including: receiving the current specification request sent by the labeling application through the entry protocol adapter, and calling the historical context session associated with the current specification request from the session context manager, wherein the current specification request includes standard application routing information and current request parameters; extracting the security identifier corresponding to the standard application from the security policy sent by the security and operation and maintenance component, and packing the current specification request, the historical context session, and the security identifier to obtain the original input information, wherein the security policy is determined by the configuration center through security configuration; and combining the original input information and the standard application routing information to determine the request information of the current session.

[0034] Specifically, in the embodiment, the receiving of the current specification request is performed by the standard application, the standard application routing information and the current request parameter are included in the current specification request, the standard application routing information can include / v1 / chat / completions, the received current request parameter conforms to the OpenAI standard specification, in addition, the entry protocol adapter also calls the historical context session associated with the current specification request from the session context manager, and the historical context session can be obtained according to the session identifier, to ensure the persistence of the session, the session context manager can also provide the creation and reuse of the session, and uniformly manage the mapping relationship between the upstream session and the backend platform session identifier, for example, the session identifier of the RAGFlow platform is session_id, the session identifier of the Dify platform is conversation_id, the session_id is maintained by the session context manager, and the conversation_id is empty when it is called for the first time, and is backfilled and bound to the upstream session by the response. Of course, in the embodiment, only an example is given, and the specific function of the session context manager is not limited. In addition, the configuration center configures the transmission of the request in the security and operation and maintenance component, the security and operation and maintenance component generates a security policy based on the security configuration, and the security policy includes the security identifier corresponding to the standard application. After obtaining the standard application routing information, the current request parameter, the historical context session and the security identifier, the above three information is packaged to obtain the original input parameter information, and the original input parameter information and the standard application routing information are combined to determine the request information of the current session.

[0035] In step S102, the standard application routing information is converted into target routing information matched with the heterogeneous RAG platform to be accessed, and the target routing information and the original input parameter information are sent to the platform request composer.

[0036] Optionally, converting the standard application routing information into target routing information matched with the heterogeneous RAG platform to be accessed comprises: determining the heterogeneous RAG platform to be accessed through the configuration center, and determining the routing format parameter of the heterogeneous RAG platform to be accessed; and converting the standard application routing information according to the routing format parameter to obtain the target routing information.

[0037] Specifically, in the embodiment, the to-be-accessed heterogeneous RAG platform is determined according to the user access demand. Since the configuration center configures the routing strategy of the to-be-accessed heterogeneous RAG platform on the entry protocol adapter, the routing format parameters matched with each heterogeneous RAG platform are included in the routing strategy. When the to-be-accessed heterogeneous RAG platform is known, the standard application routing information can be converted according to the routing format parameters corresponding to the to-be-accessed heterogeneous RAG platform to obtain target routing information adapted to the to-be-accessed heterogeneous RAG platform, so that the access to different heterogeneous RAG platforms can be realized based on the standard routing information. Therefore, the entry protocol adapter sends the obtained standard application routing information and the original input parameter information to the platform request orchestrator, and sends the original input parameter information to the to-be-accessed heterogeneous RAG platform after the platform request orchestrator processes the original input parameter information.

[0038] In step S103, the platform request orchestrator converts the original input parameter information into target input parameter information matched with the to-be-accessed heterogeneous RAG platform, and sends the target input parameter information to the to-be-accessed heterogeneous RAG platform through the target routing information.

[0039] Optionally, the platform request orchestrator converts the original input parameter information into target input parameter information matched with the to-be-accessed heterogeneous RAG platform, including: the platform request orchestrator invokes the mapping rule configured by the configuration center, wherein the mapping rule includes the corresponding relationship between the input parameter of the standard application and the parameters of each heterogeneous RAG platform; the platform request orchestrator determines the target corresponding relationship associated with the to-be-accessed heterogeneous RAG platform from the mapping rule, wherein the to-be-accessed heterogeneous RAG platform includes the RAGFlow platform or the Dify platform; and the platform request orchestrator converts the original input parameter information according to the target corresponding relationship to obtain the target input parameter information matched with the to-be-accessed heterogeneous RAG platform.

[0040] Specifically, the mapping rule is configured in advance in the platform request composer by the configuration center, and the mapping rule specifically sets the correspondence between the input parameters of the standard application and the heterogeneous RAG platform parameters, for example, the input parameters model, messages, and stream of OpenAI are mapped to the heterogeneous RAG platform parameters question, query, and conversation_id, wherein question is a RAGFlow platform parameter, query and conversation_id are Dify platform parameters. Of course, the embodiment is only an example and does not limit the parameter correspondence in the mapping rule. When the heterogeneous RAG platform to be accessed is determined, the target correspondence associated with the heterogeneous RAG platform to be accessed is determined from the mapping rule. For example, when the heterogeneous RAG platform to be accessed is a RAGFlow platform, there are 50 corresponding relationships in the mapping rule, but only 25 corresponding relationships are related to the RAGFlow platform. Therefore, the 25 corresponding relationships are used as the target correspondence. When the original input parameter information is the parameter matched by the OpenAI platform protocol, the original input parameter information can be converted based on the target correspondence to obtain the target input parameter information matched by the RAGFlow platform. In the embodiment, the request information of the OpenAI protocol can be uniformly received through the port of the standard application, and the request information of the OpenAI protocol can be converted in format by the platform request composer according to different heterogeneous RAG platforms to be accessed, so as to convert the request information matched by the heterogeneous RAG platform. Therefore, the unified upper standard application can access the heterogeneous RAG platform. Of course, the embodiment is only an example of accessing the RAGFlow platform, and the request information conversion mode of other heterogeneous RAG platforms is basically the same. The embodiment will not be described again.

[0041] In the embodiment, after the platform request composer converts the original input parameter information into the target input parameter information matched by the heterogeneous RAG platform to be accessed, the target input parameter information can be effectively identified because the target input parameter information meets the protocol specification of the heterogeneous RAG platform to be accessed. At this time, the platform request composer sends the converted target input parameter information to the heterogeneous RAG platform to be accessed through the target routing information. Of course, the embodiment is only an example of the RAGFlow platform or the Dify platform as the heterogeneous RAG platform to be accessed, and does not specifically limit the type of the heterogeneous RAG platform to be accessed.

[0042] Step S104, obtaining the response information generated by the target input information of the to-be-accessed heterogeneous RAG platform, determining the explicit response and the implicit response based on the logical mark in the response information, and extracting the explicit response to feed back to the standard application.

[0043] Optionally, before determining the explicit response and the implicit response based on the inference mark in the response information, it further includes: when determining that the type of the response information is a streaming response, sending the response information to a server-sent event (SSE) normalizer and differencer; and obtaining the response information after format conversion through the SSE normalizer and differencer.

[0044] It should be noted that in the present embodiment, after the target input information sent by the platform request composer is received by the to-be-accessed heterogeneous RAG platform, the target input information is processed and response information is generated, and at this time the generated response information can be a streaming response or a non-streaming response, and the subsequent processing mode corresponding to the different types is different. For example, when it is determined that the output of the to-be-accessed heterogeneous RAG platform is a streaming response, the SSE normalizer and differencer need to be involved, that is, the full feedback format streaming response is first converted into an incremental feedback format streaming response through the SSE normalizer and differencer, and then sent to the inference explicit and implicit extractor after format conversion, and when it is determined that the output of the to-be-accessed heterogeneous RAG platform is a non-streaming response, it does not need to be processed by the SSE normalizer and differencer but is directly sent to the inference explicit and implicit extractor, so Figure 2 The structure of the OpenAI protocol conversion gateway shown in the figure is mainly for processing the streaming response, and for the non-streaming response, the SSE normalizer and differencer need to be removed, and the non-streaming response is directly transmitted to the inference explicit and implicit extractor.

[0045] Optionally, based on the logical mark in the response information, the explicit response and the implicit response are determined, and the explicit response is extracted and fed back to the standard application, including: converting the original logical mark marked in the response information into a standard logical mark matched with the standard application through the inference explicit and implicit extractor, wherein the standard logical mark includes a thinking mark and an inference mark; the part of the response information marked with the thinking mark is taken as the explicit response, and the part of the response information marked with the inference mark is taken as the implicit response; the explicit response is sent to the session context manager through the inference explicit and implicit extractor, so that the session context manager feeds back the explicit response to the standard application through the entry protocol adapter.

[0046] Specifically, after the response information sent by the to-be-accessed heterogeneous RAG platform is directly obtained or is obtained after being processed by the SSE normalizer and differencer, the logical mark is included in the response information, but the inference mark is a special inference mark of the heterogeneous RAG platform, for example, a thinking mark<think> …< / think> And reasoning labels agent_thought, in order to realize compatibility with the target application OpenAI, these unique reasoning labels will be mapped to the standard logic labels of the OpenAI matched reasoning_content format, so although the thinking labels and reasoning labels are still included in the standard logic labels, the format of the thinking labels and reasoning labels has changed at this time, and then whether it is visible to the client can be controlled based on the reasoning strategy pre-configured by the configuration center according to the above standard logic labels, for example, when the reasoning strategy is explicit response visible to the client, the explicit response is sent to the session context manager through the reasoning display controller, and the session context manager feeds back the explicit response to the standard application through the entry protocol adapter, and displays it on the standard application.

[0047] It should be noted that the session context manager of the embodiment also establishes a mapping from the upstream session ID to the backend platform session ID, which is returned by the backend and is persistent at the first request without session, and provides an expiration strategy through TTL / LRU and supports multi-tenant isolation and auditing. In addition, concurrent processing can also be performed in the embodiment, that is, the standard application simultaneously receives multiple request information, and the full link uses an asynchronous HTTP client httpx.AsyncClient, and aiter_lines() is used for streaming, including timeout, exponential backoff retry, fuse and backup routing, client disconnection detection and heartbeat. In addition, the security and operation and maintenance components of the embodiment also integrate key management, CORS, access logs, full link tracking (request_id / trace_id), quota flow control, fuse retry, and monitoring index reporting, etc. Of course, the embodiment is only used as an example, and the specific functions of each component involved in the OpenAI protocol conversion gateway are not limited.

[0048] The technical scheme of the embodiment of the application, by unified open standard application, the request information of the current session is obtained, and the request information is converted into the request information matched with the heterogeneous RAG platform to be accessed through the OpenAI protocol conversion gateway, so as to realize the access to each heterogeneous RAG platform, without the need to develop codes for each heterogeneous RAG platform, thereby reducing the development cost, and the response feedback by each heterogeneous platform is converted according to the format of the labeled application, so that it can be displayed in the field of the standard application, thereby improving the access effect of each heterogeneous RAG platform and the consistency and stability of the upper application.

[0049] Embodiment two

[0050] Figure 3is a flowchart of an access method of a heterogeneous retrieval enhancement generation RAG platform provided by an embodiment of the present application, and based on the above-mentioned embodiment, before determining the explicit response and the implicit response based on the inference mark in the response information, the embodiment further includes: when determining that the type of the response information is streaming response, sending the response information to a server-sent event (SSE) normalizer and differentiator; and acquiring response information after format conversion by the SSE normalizer and differentiator through format conversion of the response information. The method includes:

[0051] In step S201, request information of a current session is acquired, and the request information includes original input parameter information and standard application routing information.

[0052] Optionally, the request information of the current session is acquired by: receiving a current specification request sent by a labeling application through an entry protocol adapter, and calling a historical context session associated with the current specification request from a session context manager, wherein the current specification request includes standard application routing information and current request parameters; extracting a security identifier corresponding to the standard application from a security policy sent by a security and operation and maintenance component, and packing the current specification request, the historical context session, and the security identifier to acquire original input parameter information, wherein the security policy is determined by a configuration center through security configuration; and combining the original input parameter information and the standard application routing information to determine the request information of the current session.

[0053] In step S202, the standard application routing information is converted into target routing information matched with a heterogeneous RAG platform to be accessed, and the target routing information and the original input parameter information are sent to a platform request composer.

[0054] Optionally, the standard application routing information is converted into target routing information matched with a heterogeneous RAG platform to be accessed, including: determining the heterogeneous RAG platform to be accessed through a configuration center, and determining routing format parameters of the heterogeneous RAG platform to be accessed; and converting the standard application routing information according to the routing format parameters to acquire the target routing information.

[0055] In step S203, the original input parameter information is converted into target input parameter information matched with the heterogeneous RAG platform to be accessed by the platform request composer, and the target input parameter information is sent to the heterogeneous RAG platform to be accessed through the target routing information.

[0056] Optionally, the platform request composer converts the original input information into target input information matched with the heterogeneous RAG platform to be accessed, including: the platform request composer invokes the mapping rule configured by the configuration center, wherein the mapping rule includes the correspondence between the input of the standard application and the parameters of each heterogeneous RAG platform; determining the target correspondence associated with the heterogeneous RAG platform to be accessed from the mapping rule, wherein the heterogeneous RAG platform to be accessed includes the RAGFlow platform or the Dify platform; and converting the original input information according to the target correspondence to obtain the target input information matched with the heterogeneous RAG platform to be accessed.

[0057] Step S204, obtaining the response information generated by the heterogeneous RAG platform to be accessed for the target input information.

[0058] Step S205, when determining that the type of the response information is a streaming response, sending the response information to the server sending event and SSE normalizer and differencer.

[0059] Step S206, the SSE normalizer and differencer format-convert the response information to obtain the format-converted response information.

[0060] Optionally, the SSE normalizer and differencer format-convert the response information to obtain the format-converted response information, including: the SSE normalizer and differencer obtain the response text and the end symbol in the response information; when determining that the response text is in full feedback format, then converting the response text into incremental format to obtain the converted response text; converting the end symbol into a standard termination signal matched with the standard application; and constructing the converted response information according to the converted response text and the standard termination signal.

[0061] Optionally, when determining that the response text is in full feedback format, then converting the response text into incremental format to obtain the converted response text, including: when determining that the accumulated text in the response text is valid and starts with the last session text, then determining that the response text is in full feedback format; the SSE normalizer and differencer perform differential calculation on the accumulated text and the last session text to obtain incremental content; and updating the accumulated text with the incremental content to obtain updated accumulated text, wherein the updated accumulated text is in incremental feedback format; and when determining that the termination signal is received, then taking the updated accumulated text as the converted response text.

[0062] Optionally, the method further comprises: when it is determined that the accumulated text in the response text is valid and not the above last session text as the beginning, then determining that the response text is in the incremental feedback format; when the text rollback or truncation abnormal situation is detected by the SSE normalization and the differentiator, then determining the incremental content by using the strategy matched with the abnormal situation; updating the accumulated text by using the incremental content to obtain updated accumulated text, and taking the updated accumulated text as the converted response text when the termination signal is received.

[0063] Specifically, as shown in the flow request timing diagram, through the mutual interaction between the structures in the OpenAI protocol conversion gateway, Figure 4 Figure 4 the processing of the request and the feedback of the response are realized, Figure 4 In the embodiment, only the processing of the flow request is taken as an example for description, while when it is a non-flow request, only the SSE normalization and the differentiator need to be deleted, and the contents interacting with them need to be deleted. In the embodiment, the details are not described again. In addition, in the embodiment, after the response text and the end symbol in the response information are obtained, when it is determined that the response text is the full amount feedback text, since it does not match the incremental feedback format of the standard application OpenAI, the response text is converted according to the incremental format to obtain the converted response text. In addition, the end symbol is converted into the standard termination signal matched with the standard application to realize the compatibility of data: [DONE], finish_reason / event: message_end / done, and the like, and the converted response information is constructed according to the converted response text and the standard termination signal.

[0064] ​The SSE normalizer and differentiator of the embodiment can perform online differential calculation on the accumulated text, output the incremental content, implement error diversion, ensure that the error information is returned through the standard HTTP status code and the JSON format, and does not pollute the data stream. The differential algorithm performs prefix difference on the accumulated text cur and the last session text last, that is, delta = cur[len(last):], and takes the delta as the incremental content. The accumulated text cur is updated to obtain the updated accumulated text, that is, last = cur. The updated cur is in the incremental feedback format. When it is determined that the termination signal is received, the updated accumulated text is taken as the conversion response text and is output. In addition, when it is determined that the response text is not started with the last session text, it is determined that the response text is in the incremental feedback format. Although the response text does not need to be converted, if an abnormal situation such as rollback or truncation is detected, the full amount or the longest common subsequence can be used to improve the robustness. Therefore, in this case, the strategy matched with the abnormal situation needs to be used to determine the incremental content delta. For example, for text rollback, the default strategy can be used to roll back the full amount delta = cur. For truncation, the advanced strategy can be used to calculate the difference through the local self-correlation (LSC) algorithm to determine the incremental content. Of course, the embodiment is only used for illustration, and the specific way of calculating the incremental content is not limited.

[0065] In step S207, the explicit response and the implicit response are determined based on the logical mark in the response information, and the explicit response is extracted and fed back to the standard application.

[0066] Specifically, in the embodiment, the unified protocol entry is proposed, and an "OpenAI compatible protocol conversion gateway" overall architecture is proposed. The heterogeneous RAG platform API is uniformly mapped in the request and streaming response directions, so that the upper layer application only needs to be compatible with the OpenAI protocol, and completely shields the differences of the downstream RAG platform. The problems of termination semantic unification, online differential calculation, error shunting and the like of multi-source streaming response are solved, and the platform-specific reasoning content is stably mapped to the standard field and the strategy is controlled. A session context manager is designed to realize the mapping and life cycle management of the cross-platform session identification of session_id / conversation_id, and the continuity of the dialogue context is ensured. The full asynchronous, high concurrency, flow limiting / fuse / retry, observability and gray routing are integrated to form a complete solution that can be directly used in the production environment. Therefore, the embodiment greatly improves the API connection and operation and maintenance efficiency, and the upper layer business only needs to connect with a set of OpenAI protocol, and the RAG platform needs to be added or replaced without modifying the business code, which is expected to reduce the adaptation and maintenance cost by more than 50%. The user experience and system stability are significantly improved, the differential incremental output reduces the network transmission amount and the front-end rendering pressure, the reasoning is visible and invisible, and the explainability demand is met. The unified error model, fuse, retry and standby routing improve the reliability and continuity of the service.

[0067] The technical scheme of the embodiment of the application obtains the request information of the current session through the unified and open standard application, converts the request information into the request information matched with the heterogeneous RAG platform to be accessed through the OpenAI protocol conversion gateway, so as to realize the access to each heterogeneous RAG platform, without separately developing the code for each heterogeneous RAG platform, thereby reducing the development cost. The response feedback from each heterogeneous platform is converted between the visible and invisible according to the format of the marked application, so that the standard application field can be displayed, the access effect of each heterogeneous RAG platform and the consistency and stability of the upper layer application are improved.

[0068] Embodiment three

[0069] Figure 5 A structure schematic diagram of an access device of a heterogeneous retrieval enhanced generation RAG platform provided by the embodiment of the application is shown in FIG. 1. Figure 5 As shown in the figure, the device comprises a request information acquisition module 310, a request information routing module 320, an original input information conversion module 330 and a response feedback module 340.

[0070] The request information acquisition module 310 is configured to acquire the request information of the current session, and the request information comprises original input information and standard application routing information.

[0071] The request information routing module 320 is configured to convert the standard application routing information into target routing information matched with the to-be-accessed heterogeneous RAG platform, and send the target routing information and the original input parameter information to the platform request composer.

[0072] The original input parameter information conversion module 330 is configured to convert the original input parameter information into target input parameter information matched with the to-be-accessed heterogeneous RAG platform through the platform request composer, and send the target input parameter information to the to-be-accessed heterogeneous RAG platform through the target routing information.

[0073] The response feedback module 340 is configured to obtain response information generated by the to-be-accessed heterogeneous RAG platform for the target input parameter information, determine the explicit response and the implicit response based on the logical mark in the response information, and extract the explicit response to feed back to the standard application.

[0074] Optionally, the request information obtaining module is configured to receive a current specification request sent by the standard application through the entry protocol adapter, and call a historical context session associated with the current specification request from the session context manager, wherein the current specification request includes standard application routing information and current request parameters.

[0075] The security policy sent by the security and operation and maintenance component is extracted to obtain the security identifier corresponding to the standard application, and the current specification request, the historical context session and the security identifier are packaged to obtain the original input parameter information, wherein the security policy is determined by the security configuration of the configuration center.

[0076] The original input parameter information and the standard application routing information are combined to determine the request information of the current session.

[0077] Optionally, the request information routing module is configured to determine the to-be-accessed heterogeneous RAG platform, and determine the routing format parameter of the to-be-accessed heterogeneous RAG platform.

[0078] The standard application routing information is converted according to the routing format parameter to obtain the target routing information.

[0079] Optionally, the original input parameter information conversion module is configured to call the mapping rule configured by the configuration center through the platform request composer, wherein the mapping rule includes the corresponding relationship between the input parameter of the standard application and the parameters of each heterogeneous RAG platform.

[0080] The target corresponding relationship associated with the to-be-accessed heterogeneous RAG platform is determined from the mapping rule, wherein the to-be-accessed heterogeneous RAG platform includes the RAGFlow platform or the Dify platform.

[0081] The original input parameter information is converted according to the target corresponding relationship to obtain the target input parameter information matched with the to-be-accessed heterogeneous RAG platform.

[0082] Optionally, the apparatus further comprises a response information conversion module configured to, when determining that the type of the response information is streaming response, send the response information to a server-sent event (SSE) normalizer and differencer.

[0083] The response information is converted by the SSE normalizer and differencer to obtain converted response information.

[0084] Optionally, the response information conversion module is further configured to obtain the response text and an end symbol in the response information by the SSE normalizer and differencer.

[0085] When determining that the response text is in full feedback format, the response text is converted to incremental format to obtain converted response text.

[0086] The end symbol is converted into a standard termination signal matched with a standard application.

[0087] The converted response information is constructed according to the converted response text and the standard termination signal.

[0088] Optionally, the response information conversion module is further configured to, when determining that the accumulated text in the response text is valid and starts with the last session text, determine that the response text is in full feedback format.

[0089] The accumulated text and the last session text are differentially calculated by the SSE normalizer and differencer to obtain incremental content.

[0090] The accumulated text is updated by the incremental content to obtain updated accumulated text, wherein the updated accumulated text is in incremental feedback format.

[0091] When determining that the termination signal is received, the updated accumulated text is taken as the converted response text.

[0092] Optionally, the response information conversion module is further configured to, when determining that the accumulated text in the response text is valid and does not start with the last session text, determine that the response text is in incremental feedback format.

[0093] When detecting, by the SSE normalizer and differencer, that there is a text rollback or truncation abnormality, a strategy matched with the abnormality is used to determine the incremental content.

[0094] The accumulated text is updated by the incremental content to obtain updated accumulated text, and the updated accumulated text is taken as the converted response text when the termination signal is received.

[0095] Optionally, a response feedback module is configured to convert the original logical mark labeled by the response information into a standard logical mark matched with the standard application by the reasoning explicit and implicit extractor, wherein the standard logical mark comprises a thinking mark and a reasoning mark.

[0096] The part of the response information labeled with the thinking mark is regarded as an explicit response, and the part of the response information labeled with the reasoning mark is regarded as an implicit response.

[0097] The explicit response is sent to the session context manager by the reasoning explicit and implicit extractor, so that the session context manager feeds back the explicit response to the standard application through the portal protocol adapter.

[0098] The access device of the heterogeneous retrieval augmented generation (RAG) platform provided by the embodiment of the present application can execute the access method of the heterogeneous retrieval augmented generation (RAG) platform provided by any embodiment of the present application, and has the corresponding function modules and beneficial effects of the execution method.

[0099] Embodiment four

[0100] Figure 6 A structural schematic diagram of a terminal device 10 that can be used to implement embodiments of the present application is shown. The terminal device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The terminal device can also represent various forms of mobile devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit implementations of the applications described and / or claimed in this document.

[0101] The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit implementations of the applications described and / or claimed in this document.

[0102] As shown in Figure 6 The terminal device 10 includes at least one processor 11, and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is in communication connection with the at least one processor 11, wherein the memory stores a computer program that can be executed by the at least one processor. The processor 11 can execute various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the terminal device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0103] A plurality of components in the terminal device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, and the like; an output unit 17, such as various types of displays, speakers, and the like; a storage unit 18, such as a magnetic disk, an optical disk, and the like; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 19 allows the terminal device 10 to exchange information / data with other terminal devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0104] The processor 11 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, and the like. The processor 11 performs various methods and processes described above, such as the access method of the heterogeneous retrieval augmentation generation (RAG) platform.

[0105] In some embodiments, the access method of the heterogeneous retrieval augmentation generation (RAG) platform can be implemented as a computer program tangibly embodied in a computer readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed onto the terminal device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded onto the RAM 13 and executed by the processor 11, one or more steps of the access method of the heterogeneous retrieval augmentation generation (RAG) platform described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform the access method of the heterogeneous retrieval augmentation generation (RAG) platform by any other appropriate means, such as by means of firmware.

[0106] Various implementations of the apparatuses and techniques described above herein can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a complex programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable computing device including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage device, at least one input device, and at least one output device.

[0107] Computer programs implementing the access method of the RAG platform for implementing the isomer search enhancement generation of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the computer program running on the processor implements the functions / operations specified in the flow diagrams and / or block diagrams. The computer program can execute entirely on a machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

[0108] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store the computer program for use by or in connection with an instruction execution device, apparatus, or terminal device. A computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or terminal device, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium will include one or more lines of a program of instructions in a transitory signal, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage terminal device, a magnetic storage terminal device, or any suitable combination of the foregoing.

[0109] To provide for interaction with a user, the devices and techniques described here can be implemented on a terminal device having a display device (e.g., a touch screen) for displaying information to the user and a keyboard, a mouse, or a touch screen by which the user can provide input to the terminal device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0110] It should be understood that the various forms of flow shown above can be re-ordered, added to, or deleted from without departing from the spirit of the present application. For example, the steps recited in the present application can be performed in parallel, in series, or in a different order, without departing from the desired results of the technical solutions of the present application, which are not limited herein.

[0111] The above detailed description does not limit the scope of the application. Various modifications, combinations, sub-combinations and alternatives can be made to the detailed description. Any modification, equivalent replacement and improvement etc. made within the spirit and principle of the application shall be included in the scope of the application.

Claims

1. A method for accessing a heterogeneous retrieval-enhanced RAG platform, applied to an OpenAI protocol conversion gateway, characterized in that, The method includes: Obtain the request information of the current session, which includes the original input parameter information and standard application routing information; The standard application routing information is converted into target routing information that matches the heterogeneous RAG platform to be accessed, and the target routing information and the original input parameter information are sent to the platform request orchestrator. The platform requests the orchestrator to convert the original input parameter information into target input parameter information that matches the heterogeneous RAG platform to be accessed, and then sends the target input parameter information to the heterogeneous RAG platform to be accessed through the target routing information. Obtain the response information generated by the heterogeneous RAG platform to be accessed in response to the target input parameter information, determine the explicit response and implicit response based on the logical tags in the response information, and extract the explicit response and feed it back to the standard application.

2. The method according to claim 1, characterized in that, The request to obtain the current session information includes: The system receives the current specification request sent by the annotation application through the ingress protocol adapter and retrieves the historical context session associated with the current specification request from the session context manager. The current specification request includes the standard application routing information and the current request parameters. The security identifier corresponding to the standard application is extracted according to the security policy sent by the security and operability components, and the current specification request, the historical context session and the security identifier are packaged to obtain the original input parameter information, wherein the security policy is determined by the configuration center through security configuration; The original input parameter information and the standard application routing information are combined to determine the request information of the current session.

3. The method according to claim 1, characterized in that, The step of converting the standard application routing information into target routing information that matches the heterogeneous RAG platform to be accessed includes: Identify the heterogeneous RAG platform to be accessed and determine the routing format parameters of the heterogeneous RAG platform to be accessed; The standard application routing information is converted according to the routing format parameters to obtain the target routing information.

4. The method according to claim 1, characterized in that, The step of converting the original input parameter information into target input parameter information that matches the heterogeneous RAG platform to be accessed through the platform request orchestrator includes: The platform requests the orchestrator to retrieve the mapping rules configured in the configuration center, wherein the mapping rules include the correspondence between the inputs of the standard application and the parameters of each heterogeneous RAG platform; The target correspondence associated with the heterogeneous RAG platform to be accessed is determined from the mapping rules, wherein the heterogeneous RAG platform to be accessed includes the RAGFlow platform or the Dify platform; The original input parameter information is transformed according to the target correspondence to obtain the target input parameter information that matches the heterogeneous RAG platform to be accessed.

5. The method according to claim 1, characterized in that, Before determining the explicit and implicit responses based on the inference tags in the response information, the method further includes: When the type of the response information is determined to be a streaming response, the response information is sent to the server to send the event SSE normalization and differencer; The response information is format-converted by the SSE normalization and differencer to obtain the format-converted response information.

6. The method according to claim 5, characterized in that, The step of converting the response information using the SSE normalizer and differencer to obtain the format-converted response information includes: The response text and terminator in the response information are obtained through the SSE normalization and differencer. When it is determined that the response text is in full feedback format, the response text is converted according to incremental format to obtain converted response text; The terminator is converted into a standard termination signal that matches the standard application; The converted response information is constructed based on the converted response text and the standard termination signal.

7. The method according to claim 6, characterized in that, When it is determined that the response text is in full feedback format, the response text is converted according to incremental format to obtain converted response text, including: When it is determined that the cumulative text in the response text is valid and begins with the text of the previous session, the response text is determined to be the full feedback format. The incremental content is obtained by performing differential calculation on the cumulative text and the previous session text through the SSE normalization and differencer. The incremental content is used to update the cumulative text to obtain updated cumulative text, wherein the updated cumulative text is in incremental feedback format; When a termination signal is received, the updated cumulative text is used as the conversion response text.

8. The method according to claim 6, characterized in that, The method further includes: If it is determined that the cumulative text in the response text is valid and does not begin with the text of the previous session, then the response text is determined to be in incremental feedback format. When a text rollback or truncation anomaly is detected by the SSE normalization and differencer, the incremental content is determined by a strategy matching the anomaly. The incremental content is used to update the cumulative text to obtain the updated cumulative text, and the updated cumulative text is used as the conversion response text when a termination signal is received.

9. The method according to claim 2, characterized in that, The process of determining explicit and implicit responses based on logical markers in the response information, and extracting the explicit responses and feeding them back to the standard application, includes: The original logical tags annotated with the response information are converted into standard logical tags that match the standard application by the reasoning explicit-implicit extractor, wherein the standard logical tags include thinking tags and reasoning tags; The portion of the response information marked with a thinking tag is taken as the explicit response, and the portion of the response information marked with a reasoning tag is taken as the implicit response. The explicit response is sent to the session context manager via the inference explicit extractor, so that the session context manager feeds back the explicit response to the standard application through the ingress protocol adapter.

10. An access device for a heterogeneous search-enhanced generation RAG platform, characterized in that, The device includes: The request information acquisition module is used to acquire the request information of the current session, which includes the original input parameter information and standard application routing information. The request information routing module is used to convert the standard application routing information into target routing information that matches the heterogeneous RAG platform to be accessed, and send the target routing information and the original input parameter information to the platform request orchestrator. The raw input parameter information conversion module is used to convert the raw input parameter information into target input parameter information that matches the heterogeneous RAG platform to be accessed through the platform request orchestrator, and send the target input parameter information to the heterogeneous RAG platform to be accessed through the target routing information. The response feedback module is used to obtain the response information generated by the heterogeneous RAG platform to be accessed in response to the target input parameter information, determine the explicit response and implicit response based on the logical tags in the response information, and extract the explicit response and feed it back to the standard application.

11. A terminal device, characterized in that, The terminal device includes: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-9.

12. A storage medium for computer-executable instructions, wherein a computer program is stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-9.

Images