Updating method and device for retrieval enhanced knowledge base, medium and product

Abstract

The invention discloses an updating method and device for retrieving an enhanced knowledge base, a medium and a product, and the method comprises the steps: monitoring a data increment event through a monitoring client, generating an increment message, and sending the increment message to message middleware; storing the data increment message received in real time to a message queue through the message middleware; obtaining a message through the synchronization main service, and calling a matched plug-in to process the message; analyzing the data through the synchronization plug-in, and feeding back knowledge base operation data to the synchronization main service; when a synchronization main service receives knowledge base operation data, data synchronization is carried out on a retrieval enhancement knowledge base matched with the knowledge base operation data for large language model retrieval enhancement generation, and the technical scheme of the embodiment of the invention realizes automatic circulation from business data to retrieval enhancement knowledge; the timeliness and accuracy of updating of the retrieval enhancement knowledge base are greatly improved, and the problems that a traditional manual maintenance mode is low in efficiency and prone to making mistakes are effectively solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a retrieval augmented knowledge base updating method, device, medium and product. BACKGROUND

[0002] Retrieval augmented generation (RAG) technology refers to storing domain documents when a large language model is used for natural language question answering by using a retrieval augmented knowledge base. Specifically, the large language model first retrieves a plurality of matching documents in the corresponding retrieval augmented knowledge base according to the user input question, and then generates an answer matching the question based on the above matching documents, so as to improve the accuracy and timeliness of the large language model output.

[0003] In related technologies, a large language model service provider can maintain different retrieval augmented databases for different business parties to provide personalized and customized retrieval augmented services to each business party. In order to ensure the timeliness of the retrieval augmented database, the business party can store real-time updated business data in a business database. Then, the structured business data can be periodically exported from the business database by manual means, and the field mapping rules are manually configured for the above structured business data and manually imported into the corresponding retrieval augmented database to realize dynamic maintenance of the retrieval augmented database.

[0004] The inventors have found in the process of implementing the present application that although this manual implementation method is intuitive, when the business data frequently changes, a large amount of manual cost needs to be introduced to ensure the timeliness of the retrieval augmented knowledge base, and at the same time, the complex field mapping rule configuration process is prone to errors, and once the wrong data enters the retrieval augmented knowledge base, it will directly affect the output quality of the large language model. SUMMARY

[0005] The present application provides a retrieval augmented knowledge base updating method, device, medium and product, which can update the retrieval augmented knowledge base in the large language model in real time as the business data changes without introducing any manual cost.

[0006] According to an aspect of an embodiment of the present application, a retrieval augmented knowledge base updating method is provided, which is executed by a knowledge base synchronization system including a listening client, a message middleware, a synchronization master service and a plurality of synchronization plug-ins, and the method comprises:

[0007] When the listening client listens to a data increment event of a business data table, a data increment message matching the data increment event is generated, and the data increment message is sent to the message middleware;

[0008] The real-time received data incremental message is stored to the message queue through the message middleware;

[0009] The message is sequentially obtained from the message queue by the synchronization main service, and the synchronization plug-in matched with the obtained message is called in each synchronization plug-in for message processing;

[0010] The knowledge base operation data is obtained by the synchronization plug-in in response to the calling of the synchronization main service, and the knowledge base operation data is fed back to the synchronization main service;

[0011] The retrieval enhanced knowledge base matched with the knowledge base operation data is synchronized by the synchronization main service when the knowledge base operation data is received, wherein the retrieval enhanced knowledge base is used when the large language model is used for retrieval enhancement generation.

[0012] According to another aspect of the embodiment of the application, an updating device of a retrieval enhanced knowledge base is provided, which is configured in a knowledge base synchronization system including a listening client, a message middleware, a synchronization main service and a plurality of synchronization plug-ins, and the device includes:

[0013] The listening module is configured to generate a data incremental message matched with a data incremental event and send the data incremental message to the message middleware when the listening client listens to the data incremental event of the business data table;

[0014] The transmission module is configured to store the real-time received data incremental message to the message queue through the message middleware;

[0015] The scheduling module is configured to sequentially obtain the message from the message queue by the synchronization main service, and call the synchronization plug-in matched with the obtained message in each synchronization plug-in for message processing;

[0016] The conversion module is configured to obtain the knowledge base operation data by the synchronization plug-in in response to the calling of the synchronization main service, and feed back the knowledge base operation data to the synchronization main service;

[0017] The updating module is configured to synchronize the retrieval enhanced knowledge base matched with the knowledge base operation data by the synchronization main service when the knowledge base operation data is received, wherein the retrieval enhanced knowledge base is used when the large language model is used for retrieval enhancement generation.

[0018] According to another aspect of the embodiment of the application, an electronic device is provided, which includes:

[0019] At least one processor; and

[0020] The memory is in communication connection with the at least one processor; wherein,

[0021] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the method for searching an updated enhanced knowledge base according to any one of the embodiments of the present application.

[0022] According to another aspect of the embodiments of the present application, a computer readable storage medium is provided, which stores computer instructions for enabling a processor to implement the method for searching an updated enhanced knowledge base according to any one of the embodiments of the present application when the processor executes the computer instructions.

[0023] According to another aspect of the embodiments of the present application, a computer program product is also provided, which comprises computer instructions for enabling a processor to implement the steps of the method according to any one of the embodiments of the present application when the processor executes the computer instructions.

[0024] The technical solution of the embodiments of the present application provides a new type of method for updating an enhanced knowledge base by generating a data increment message matched with a data increment event when the client listens to a business data table and sends the data increment message to a message middleware, storing the real-time received data increment message to a message queue through the message middleware, sequentially obtaining the message from the message queue through a synchronization master service, and calling a synchronization plug-in matched with the obtained message in each synchronization plug-in for message processing, and by responding to the calling of the synchronization master service through the synchronization plug-in, the synchronization master service sends the message, the message is parsed to obtain knowledge base operation data, and the knowledge base operation data is fed back to the synchronization master service, and by implementing the data synchronization of the enhanced knowledge base matched with the knowledge base operation data through the synchronization master service when the knowledge base operation data is received, the method for updating the enhanced knowledge base is provided, which realizes the automatic update of the business data to the enhanced knowledge base without introducing any human cost, greatly improves the timeliness and accuracy of the update of the enhanced knowledge base, and has good expansibility and stability through the plug-in architecture and the standardized processing flow, and can flexibly adapt to the data synchronization requirements of different business scenarios.

[0025] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present application, nor to limit the scope of the present application. Other features of the present application will become apparent through the following description. BRIEF DESCRIPTION OF DRAWINGS

[0026] In order to make the technical solution in the embodiments of the present application clearer, the accompanying drawings needed in the embodiment description will be briefly introduced. Obviously, the accompanying drawings in the following description are only some embodiments of the present application, and all other embodiments obtained by those skilled in the art without any creative effort should belong to the protection scope of the present application.

[0027] Figure 1 is a flow chart of a method for searching an enhanced knowledge base according to an embodiment of the present application;

[0028] Figure 2 is a flow chart of another method for searching an enhanced knowledge base according to another embodiment of the present application;

[0029] Figure 3 is a flow chart of another method for searching an enhanced knowledge base according to another embodiment of the present application;

[0030] Figure 4 is a flow chart of another method for searching an enhanced knowledge base according to another embodiment of the present application;

[0031] Figure 5 is a structural schematic diagram of an apparatus for searching an enhanced knowledge base according to an embodiment of the present application;

[0032] Figure 6 is a structural schematic diagram of an electronic device for searching an enhanced knowledge base according to an embodiment of the present application. DETAILED DESCRIPTION

[0033] In order to make the technical solution in the embodiments of the present application clearer, the accompanying drawings needed in the embodiment description will be briefly introduced. Obviously, the accompanying drawings in the following description are only some embodiments of the present application, and all other embodiments obtained by those skilled in the art without any creative effort should belong to the protection scope of the present application.

[0034] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and in the above drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not have to be limited to only those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0035] Embodiment one

[0036] Figure 1 A flowchart of a method for updating a retrieval enhancement knowledge base is provided for the first embodiment of the present application. The present embodiment can be applicable to the case where the retrieval enhancement knowledge base in the large language model is automatically triggered to update in real time following the changes in business data. The method can be performed by an updating device for the retrieval enhancement knowledge base, which can be realized in the form of hardware and / or software and can generally be configured in a knowledge base synchronization system. The knowledge base synchronization system includes a listening client, a message middleware, a synchronization master service and a plurality of synchronization plug-ins. The above knowledge base synchronization system can be configured in a single electronic device or in a service cluster composed of multiple electronic devices, and the present embodiment does not limit this.

[0037] Correspondingly, as shown in Figure 1 , the method comprises:

[0038] S110, when a data increment event of a business data table is listened to by the listening client, a data increment message matched with the data increment event is generated, and the data increment message is sent to the message middleware.

[0039] The listening client can be understood as a specially designed database change capture component, which uses a mechanism similar to a MySQL slave to capture data changes in real time by parsing the binary log of the database. The component is deployed in a lightweight manner on the database server or intermediate node, continuously monitoring the write operation of the specified business table. The business data table can be understood as a structured data table that stores core business data in enterprise applications, usually located in a relational database. These tables contain customer information, transaction records or product catalogs and other key business entities, with clear table structure definition and standardized data relationship.

[0040] In an optional implementation of the embodiment, the monitoring client can be constructed by an open-source stream processing server and a client plug-in with message processing capability. The business data table can be understood as a data table maintained by a business party accessing the large language model, in which business data for updating the retrieval enhancement knowledge in the retrieval enhancement knowledge base is stored.

[0041] Specifically, the large language model (LLM) can be understood as a pre-trained model based on a deep neural network architecture (e.g., Transformer) with a large number of model parameters (e.g., more than 1 billion), which can be a generative system, i.e., after training on massive data, it can generate new content based on natural language understanding, generation and reasoning for multi-source input data such as text, image or audio / video. Typically, the large language model can be a self-recursive language model that supports setting the context size.

[0042] The data increment event can be understood as a specific change operation of the business data table, mainly including three basic types: adding a record, updating a record and deleting a record. Each event contains complete change context, such as operation timestamp, table name, primary key value and field value before and after change, etc. The message middleware plays the role of data transmission hub in the scheme, responsible for reliably delivering the data increment message matched with the data increment event to the downstream processing link after the data increment event is generated.

[0043] In the embodiment, the monitoring client continuously monitors the state change of the business data table, and immediately generates a data increment message containing complete change context when detecting a data increment event such as adding, updating or deleting, which accurately records the operation type, table name, primary key value and change field content, etc. The generated increment message is transmitted through the message middleware to ensure that the change information can be reliably delivered to the downstream processing link.

[0044] In an optional implementation of the embodiment, the same large language model service provider can provide natural language question and answer services for multiple business parties. Each business party independently maintains its own business data table.

[0045] Correspondingly, the number of monitoring clients can be unique, and the above unique monitoring client can monitor the data increment events of different business data tables maintained by different business parties. Alternatively, the number of the above monitoring clients can correspond to the number of business parties one by one, and different monitoring clients are used to monitor the data increment events of different business data tables maintained by different business parties.

[0046] S120, store the real-time received data increment message to the message queue through the message middleware.

[0047] The message queue can be understood as a transmission channel of the data change event, and is responsible for reliably delivering the change information between the listening client and the synchronization master service.

[0048] In the embodiment, the message middleware stores the data incremental messages from the listening client in the pre-configured message queue immediately, and ensures the message sequence by using the first-in first-out processing principle. The message middleware can be implemented based on STAN (NATS Streaming Server, NATS streaming server), and the distributed architecture and high-availability design of the STAN can ensure that the messages are not lost or repeated even in the case of network fluctuations or high load.

[0049] S130, sequentially obtaining the messages from the message queue by the synchronization master service, and calling the synchronization plug-in matched with the obtained message in each synchronization plug-in for message processing.

[0050] The synchronization master service is the core coordination component of the scheme, undertakes the central scheduling responsibility of the data synchronization process, and is responsible for accurately synchronizing the business data changes to the retrieval enhanced knowledge base. The synchronization plug-in can be understood as a core processing unit specially designed for converting the messages (i.e., data incremental messages) in the message queue into knowledge base operation data. As a "translator" between the business data and the retrieval enhanced knowledge base of the large language model, the synchronization plug-in realizes the intelligent conversion between different forms of data. In short, the synchronization master service can be understood as an executable code set that can directly operate (update) the retrieval enhanced knowledge base. By executing the synchronization master service, various knowledge base update plug-ins provided by the retrieval enhanced knowledge base can be called to perform operations such as deleting, newly creating, or deleting knowledge data on the retrieval enhanced knowledge base.

[0051] As described above, when the large language model accesses multiple business parties, and each business party independently maintains its own business data table, the data formats of the business data tables of different business parties, or the field naming of the same meaning field value (for example, for the specific name field value, business data table 1 uses “NAME” as the field name, and business data table 2 uses “name1” as the field name) are different. Therefore, it is necessary to pre-register the synchronization plug-ins corresponding to different business data tables in the synchronization master service, for converting the data incremental message matching the business data table of a specific format into the knowledge base operation data recognizable by the retrieval enhanced knowledge base. In the embodiment, after the synchronization master service sequentially obtains the data incremental message from the message queue, the corresponding synchronization plug-in is accurately matched through the pre-established plug-in registration table, which records the business data table identifier and operation type that can be processed by each plug-in; during the matching process, double verification is performed according to the table name and event type in the message to ensure that the correct processing plug-in is selected; then the message processing interface of the target plug-in is called, and the incremental message containing the complete change context is transmitted to the plug-in, which at least includes the operation type, table name, primary key value and change field content and other metadata.

[0052] S140, the synchronization plug-in responds to the call of the synchronization master service to parse the knowledge base operation data obtained from the message sent by the synchronization master service, and feeds back the knowledge base operation data to the synchronization master service.

[0053] The knowledge base operation data can be understood as operation data that can be recognized by the synchronization master service and used to update the retrieval enhanced knowledge base. The retrieval enhanced knowledge base can be understood as a retrieval enhanced knowledge storage system constructed based on an open source large language model application development platform, which is a structured knowledge storage system specially designed to support large language model retrieval enhancement generation.

[0054] In the embodiment, after the synchronization plug-in receives the data incremental message transmitted by the synchronization master service, the operation type, table name and change field content and other metadata are first parsed from the message body; for the delete event, a delete instruction containing the data source primary key is directly generated, and for the add and modify events, the structured row data is converted into natural language format knowledge content based on the column name and column annotation of the data table; during the conversion process, unnecessary fields are filtered according to the list of unnecessary fields configured by the plug-in, and the keywords column is extracted as the retrieval keyword; after the conversion is completed, the standardized knowledge base operation data is generated, including the operation type, target knowledge base identifier, document identifier, data source primary key and knowledge content and other key fields; finally, these operation data are fed back to the synchronization master service through the defined interface specification, and the entire processing process strictly follows the pre-set field mapping relationship and content assembly rule, to ensure that the output knowledge base operation data conforms to the knowledge base calling specification.

[0055] S150, synchronizing the retrieval-enhanced knowledge base matched with the knowledge base operation data when receiving the knowledge base operation data through the synchronization master service, wherein the retrieval-enhanced knowledge base is used when the large language model generates retrieval enhancement.

[0056] The retrieval-enhanced knowledge base can be understood as a structured knowledge storage system specially designed to support the retrieval enhancement generation of the large language model. By converting business data into machine-understandable semantic knowledge, the AI (Artificial Intelligence) model provides accurate information retrieval basis.

[0057] The large language model can correspond to multiple retrieval-enhanced knowledge bases, each retrieval-enhanced knowledge base can include multiple documents, and each document can include multiple knowledge segments. Furthermore, the retrieval-enhanced knowledge base stores knowledge data in the form of knowledge segments as the smallest unit.

[0058] In this embodiment, after the synchronization master service receives the knowledge base operation data fed back by the synchronization plug-in, it first queries the document synchronization mapping table according to the data source name and the source data primary key to obtain the corresponding relationship. When the operation type is delete, the corresponding segment in the specified data set is deleted through the knowledge base interface, and the related records in the mapping table are removed. When the operation type is add and the mapping relationship does not exist, the knowledge base interface is called to add a segment in the specified document, and a new mapping relationship is established. When the operation type is modify and the mapping relationship exists, the knowledge base interface is called to update the existing segment content. All operations are completed through standardized knowledge base interfaces, and the interface request contains necessary parameters such as data set identifier, document identifier and segment identifier, ensuring accurate positioning of the target knowledge segment. After each operation is completed, the synchronization master service updates the mapping table record in time to maintain the correspondence between the data source primary key and the knowledge base segment identifier, providing accurate positioning basis for subsequent synchronization operations.

[0059] The technical solution of this invention involves a monitoring client generating a data increment message matching the data increment event when it detects a data increment event in a business data table. This data increment message is then sent to a message middleware, which stores the received data increment message in real-time into a message queue. A synchronization master service sequentially retrieves messages from the message queue and calls the corresponding synchronization plugin within each synchronization plugin for message processing. The synchronization plugins respond to the calls from the synchronization master service by parsing the messages sent by the master service to obtain knowledge base operation data, which is then fed back to the master service. Upon receiving the knowledge base operation data, the master service synchronizes the data with the retrieval-enhanced knowledge base that matches it. This novel method for updating the retrieval-enhanced knowledge base achieves automated updates from business data to the knowledge base, significantly improving the timeliness and accuracy of knowledge base updates. Furthermore, its pluggable architecture and standardized processing flow provide excellent scalability and stability, enabling flexible adaptation to the data synchronization needs of different business scenarios.

[0060] Furthermore, based on the above embodiments, before calling the synchronization plugin that matches the acquired message in each synchronization plugin through the main synchronization service to process the message, it may further include:

[0061] Each synchronization plugin registers with the main synchronization service based on the business data table it processes, the message processing type of the plugin, and the retrieval enhancement knowledge base that the plugin needs to synchronize.

[0062] Generally, synchronization plugins need to complete a registration process with the main synchronization service. This is because different business data tables have their own specific data structure formats, requiring dedicated synchronization plugins to convert these formats to the knowledge base standard format. Each synchronization plugin must explicitly declare three core pieces of information during registration: the scope of the business data tables it is responsible for, the types of messages it supports (such as add, modify, or delete), and the corresponding target knowledge base identifier. This registration information forms the routing basis for the subsequent data synchronization process.

[0063] After registration, the main synchronization service will create a complete plugin registration information table. This table records in detail the correspondence between plugins and business data tables, operation types, and knowledge bases. When processing data change messages, the main synchronization service can quickly locate the matching plugin instance by querying this registry based on the two key fields of table name and operation type in the message.

[0064] The registration mechanism ensures that different types of business data changes are handled by the correct plug-in and synchronized to the designated knowledge base, while supporting dynamic expansion of new business data table synchronization requirements. Only the corresponding plug-in needs to be developed and registered, without modifying the core processing logic. The entire registration process uses a standardized interface protocol. The plug-in needs to provide necessary configuration parameters, including but not limited to data table name, supported operation type list, and target knowledge base identifier, etc. These information will be used for subsequent message routing and processing decisions.

[0065] Optionally, based on the above embodiments, the knowledge base operation data obtained by the synchronization plug-in in response to the call of the synchronization master service from the message sent by the synchronization master service can include:

[0066] The target message sent by the synchronization master service is parsed by the target synchronization plug-in in response to the call of the synchronization master service to obtain a plurality of key fields;

[0067] Among the key fields, there are multiple data including: knowledge base operation type, target search enhanced knowledge base required for synchronization, target document in the target search enhanced knowledge base required for synchronization, target data field in the target business data table required for processing, and target field value matched with the target data field;

[0068] The obtained plurality of key fields are filled into a preset standardized template to obtain knowledge base operation data matched with the target message.

[0069] Generally, when the synchronization master service initiates a call, the target synchronization plug-in will immediately respond and start processing the received message content. The plug-in first performs in-depth analysis on the message to extract multiple key information elements, which form the basis for subsequent data conversion. The analysis process focuses on five core fields: the operation type required by the knowledge base, the unique identifier of the target search enhanced knowledge base, the document identifier to be operated in the knowledge base, the field name involved in the business data table, and the specific numerical value corresponding to these fields. These fields together constitute the complete context required for data conversion.

[0070] Generally, after obtaining all the key fields, the plug-in assembles the data according to a predefined standard template. This template specifies the unified structure and format requirements of the knowledge base operation data, ensuring the consistency of the conversion results of different business tables. The plug-in will parse the obtained operation type, knowledge base identifier, document identifier, field name, and field value, and accurately fill them according to the rules defined in the template, finally generating standardized knowledge base operation data. This standardized conversion process ensures that even if faced with different business tables and data types, the output operation data can meet the calling specifications of the knowledge base interface, providing reliable data input for subsequent synchronization operations. The entire process strictly follows the preset field mapping relationship and content assembly rules, ensuring the consistency and accuracy of data processing.

[0071] Embodiment Two

[0072] Figure 2 Another flowchart of the method for retrieving an updated enhanced knowledge base is provided in Embodiment Two of the present application, which is optimized based on the above-mentioned embodiments. The knowledge base operation types include: deleting data, adding new data, and modifying data. Specifically, the operation of deleting data in the step of "synchronizing the enhanced knowledge base matching the knowledge base operation data when the knowledge base operation data is received through the synchronization master service" is refined.

[0073] Correspondingly, as shown in Figure 2 , the method comprises:

[0074] S210, when the business data table is found to have a data increment event through the listening client, a data increment message matching the data increment event is generated, and the data increment message is sent to the message middleware.

[0075] S220, the data increment message received in real time is stored in the message queue through the message middleware.

[0076] S230, the messages are sequentially obtained from the message queue through the synchronization master service, and the synchronization plug-in matching the obtained message is called in each synchronization plug-in for message processing.

[0077] S240, the knowledge base operation data is parsed from the message sent by the synchronization master service through the synchronization plug-in in response to the calling of the synchronization master service, and the knowledge base operation data is fed back to the synchronization master service.

[0078] When the knowledge base operation type in the present embodiment is deleting data, the knowledge base operation data is deleting the knowledge data matching the target data field in the target document of the target enhanced knowledge base.

[0079] S250, in the process of deleting the knowledge data matched with the target data field in the target document of the target retrieval-enhanced knowledge base by the synchronization master service through the received knowledge base operation data, obtaining a target mapping relationship table matched with the target retrieval-enhanced knowledge base from the plurality of mapping relationship tables maintained locally.

[0080] In the mapping relationship table, the mapping relationship between each data field in the set business data table and each knowledge field in the set retrieval-enhanced knowledge base is stored.

[0081] In the embodiment, when the synchronization master service processes the deletion operation, first, the specific mapping relationship table corresponding to the target retrieval-enhanced knowledge base is located from the plurality of mapping relationship tables maintained locally. The complete mapping relationship chain of the target business data table field and the knowledge base knowledge field is stored in the table, including the key mapping information such as the data source name, the data source primary key, the data set identifier, the document identifier and the segment identifier. Through the query of the table, the synchronization master service can obtain the accurate corresponding relationship between the target business data field and the knowledge base field, and provide accurate positioning basis for the subsequent deletion operation.

[0082] S260, obtaining the target knowledge field matched with the target data field and the target knowledge segment to which the target knowledge field belongs by querying the target mapping relationship table through the synchronization master service.

[0083] In the embodiment, the synchronization master service obtains the knowledge field information corresponding to the business data field to be deleted by querying the target mapping relationship table according to the received deletion operation data; the synchronization master service accurately positions the target knowledge field in the mapping table according to the data source name and the source data primary key, and obtains the specific knowledge segment identifier to which the knowledge field belongs, and these information includes the key parameters such as the data set identifier, the document identifier and the segment identifier; through the complete mapping chain, the synchronization master service can accurately determine the specific position of the knowledge content to be deleted in the retrieval-enhanced knowledge base, provide necessary position parameters for the subsequent construction of accurate deletion instruction, and ensure that the deletion operation can accurately act on the content corresponding to the target knowledge field in the specific segment of the specified document in the target knowledge base.

[0084] S270, constructing the knowledge deletion instruction for deleting the knowledge data corresponding to the target knowledge field in the target knowledge segment of the target document of the target retrieval-enhanced knowledge base through the synchronization master service.

[0085] In this embodiment, the synchronization master service constructs an accurate knowledge deletion instruction based on the acquired target knowledge base identifier, document identifier, segment identifier, and target knowledge field information. This instruction explicitly specifies the specific content location that needs to be deleted in the search-enhanced knowledge base, including three-level positioning parameters such as dataset identifier, document identifier, and segment identifier, ensuring that the deletion operation only affects the specific knowledge content associated with the business data field. The instruction generation process strictly follows the pre-set interface specification, contains complete operation type identifier and target positioning information, and is encapsulated through a standardized data format, providing clear and explicit execution parameters for subsequent calls to the knowledge base deletion interface. The construction of the deletion instruction completely relies on the accurate mapping information obtained from the mapping relationship table in the early stage, ensuring that the deletion of business data can be accurately transmitted to the associated knowledge base content, maintaining the consistency of data and knowledge.

[0086] S280, calling the plug-in interface matched with the search-enhanced knowledge base through the synchronization master service to execute the knowledge deletion instruction.

[0087] In this embodiment, the synchronization master service must interact with the search-enhanced knowledge base through a specific plug-in interface when performing the deletion operation, rather than directly operating the knowledge base data. Under this design architecture, the synchronization master service is responsible for generating standardized deletion instructions, but the actual data deletion operation is completed by a specially designed plug-in interface. This calling process strictly follows the pre-set interface specification and passes the deletion instruction containing complete positioning information such as target knowledge base identifier, document identifier, segment identifier, and target knowledge field to the knowledge base processing module. After receiving the deletion request, the knowledge base first verifies the integrity and validity of the instruction parameters, and after confirming that there is no error, it accurately locates and deletes the knowledge content associated with the target knowledge field in the specified segment of the specified document in the specified knowledge base.

[0088] S290, deleting the mapping relationship between the target data field and the target knowledge field in the target mapping relationship table through the synchronization master service.

[0089] In this embodiment, after the synchronization master service performs the knowledge base deletion operation, it immediately updates the locally maintained mapping relationship table and deletes the mapping relationship record between the business data field and the knowledge field corresponding to the completed deletion operation from the target mapping relationship table. This operation is based on the data source name and source data primary key obtained in the early stage to accurately locate the mapping entries that need to be removed, ensuring that only valid data association records are retained in the mapping relationship table. The process of deleting the mapping relationship is strictly synchronized with the knowledge base deletion operation, ensuring consistency through transaction mechanisms, avoiding abnormal situations where knowledge content has been deleted but the mapping relationship still exists.

[0090] The technical scheme of the embodiment of the application captures the incremental change event of the business data table in real time through a listening client, generates a data incremental message containing metadata such as an operation type, a table name and a field value, and sends the data incremental message to a message middleware; the message middleware stores the incremental message persistently in a message queue after receiving the incremental message, thereby ensuring the ordered delivery of the message; after the synchronization master service obtains the message from the queue, the synchronization master service calls a matched synchronization plug-in for processing according to the table name and the operation type; the synchronization plug-in analyzes the message content, converts the structured data into standardized knowledge base operation data and feeds back the knowledge base operation data; when the operation type is deletion, the synchronization master service queries a mapping relationship table to obtain a target knowledge field and segmentation information, constructs a deletion instruction and executes the deletion instruction through a plug-in interface, and finally updates the mapping relationship table. This novel method of updating the enhanced knowledge base through retrieval realizes the automatic flow of business data to the knowledge base, and greatly improves the timeliness and accuracy of the updating of the knowledge base. In particular, a complete synchronization mechanism is established for the deletion operation, so that the associated knowledge content can be accurately located and synchronized after the deletion of the business data, thereby effectively avoiding the problem of residual invalid data in the knowledge base.

[0091] Embodiment three

[0092] Figure 3 The flowchart of another method of updating the enhanced knowledge base through retrieval provided by the third embodiment of the application is optimized based on the above-mentioned embodiments. The knowledge base operation types include: deleting data, adding data and modifying data. The operation of adding data in the step of synchronizing the enhanced knowledge base through retrieval with the knowledge base operation data when the synchronization master service receives the knowledge base operation data is refined.

[0093] Correspondingly, as shown in Figure 3 , the method comprises:

[0094] S310, when the listening client listens to the occurrence of a data incremental event of a business data table, generating a data incremental message matched with the data incremental event, and sending the data incremental message to a message middleware.

[0095] S320, storing the data incremental message received in real time by the message middleware into a message queue.

[0096] S330, obtaining the messages from the message queue by the synchronization master service in sequence, and calling the synchronization plug-in matched with the obtained message in each synchronization plug-in for message processing.

[0097] S340, in response to the calling of the synchronization master service by the synchronization plug-in, analyzing the message sent by the synchronization master service to obtain knowledge base operation data, and feeding back the knowledge base operation data to the synchronization master service;

[0098] When the knowledge base operation type in the embodiment is adding data, the knowledge base operation data is adding knowledge data matching the target data field and the target field value in the target document of the target search enhanced knowledge base.

[0099] S350, acquiring, by the synchronization master service, the target knowledge segment in the target document when the received knowledge base operation data is adding knowledge data matching the target data field and the target field value in the target document of the target search enhanced knowledge base.

[0100] In the embodiment, the synchronization master service determines the specific knowledge segment position where the added content should be stored based on the target document identifier specified in the received knowledge base operation data when processing the adding operation by querying the document structure information of the knowledge base. The positioning process comprehensively considers the preset segment rules of the knowledge base, including single segment capacity limit (usually not more than a set token threshold), semantic correlation degree (to ensure content theme consistency) and adjacent segment overlap ratio requirement, and finally selects the most suitable segment as the adding target.

[0101] S360, creating, by the synchronization master service, a target knowledge field matching the target data field, and constructing target knowledge data according to the target knowledge field and the target field value.

[0102] In the embodiment, the synchronization master service creates a target knowledge field matching the target data field in the business data table according to the received knowledge base operation data and the determined target knowledge segment information, and this process strictly follows the preset field mapping rules; based on the target knowledge field and the target field value extracted from the incremental message, the standardized target knowledge data is constructed according to the content specification of the knowledge base, to ensure that the added content meets the format requirements and semantic standards of the knowledge base; the newly created knowledge field is accurately corresponding to the business data field, and meanwhile, the unique content organization method and retrieval requirement of the knowledge base are considered, and the constructed knowledge data contains complete field identifier, content value and metadata information, to provide standardized input content for the generation of subsequent adding instructions.

[0103] S370, constructing, by the synchronization master service, a knowledge adding instruction of adding the target knowledge data in the target knowledge segment of the target document of the target search enhanced knowledge base.

[0104] In this embodiment, the synchronization master service assembles and generates a standardized knowledge addition instruction based on the obtained target knowledge base identifier, document identifier, segment identifier, newly created target knowledge field, and constructed target knowledge data. The instruction explicitly includes operation type identifier, target knowledge base positioning information (data set identifier, document identifier, and segment identifier), and specific knowledge content to be added, strictly following the format and structure required by the knowledge base interface specification. The generation process of the addition instruction is completely based on the accurate mapping information obtained from the mapping relationship table in the early stage, ensuring that the business data addition can be accurately transmitted to the associated knowledge base content position, while considering the unique segmentation rules and content organization of the knowledge base, ensuring the semantic coherence and structural rationality of the added content and the existing knowledge. The instruction specifies the exact position and content details of the added knowledge content in the target knowledge base, providing complete and explicit execution parameters for subsequent calls to the knowledge base addition interface.

[0105] S380, calling the plug-in interface matched with the enhanced knowledge base for retrieval through the synchronization master service, and executing the knowledge addition instruction.

[0106] In this embodiment, after constructing the knowledge addition instruction, the synchronization master service executes the instruction by calling the plug-in interface matched with the enhanced knowledge base for retrieval. The calling process strictly follows the preset interface specification, and delivers the addition instruction containing complete parameters such as target knowledge base identifier, document identifier, segment identifier, and knowledge content to be added to the knowledge base processing module. After receiving the addition request, the knowledge base first verifies the integrity and validity of the instruction parameters, and confirms that there is no error in the specified knowledge base of the corresponding segment of the specified document, and accurately adds the knowledge content associated with the target knowledge field according to the instruction requirements.

[0107] Further, the entire execution process adopts a transaction mechanism to ensure the atomicity of the operation, that is, the addition operation and the subsequent mapping relationship update are a complete transaction unit, either all successfully executed or all rolled back, so as to maintain the consistency of business data and knowledge base content. After execution is completed, the knowledge base returns the operation result status for the synchronization master service to record and subsequent processing, while updating the relevant statistical information and index structure to ensure that the added content can be efficiently retrieved and used by the large language model.

[0108] S390, obtaining the target mapping relationship table matched with the target enhanced knowledge base for retrieval from the plurality of mapping relationship tables maintained locally by the synchronization master service, and adding the mapping relationship between the target data field and the target knowledge field in the target mapping relationship table.

[0109] In the embodiment, the synchronization master service updates the locally maintained mapping relationship table after performing the knowledge base addition operation, accurately obtains the specific mapping relationship table matched with the target search enhanced knowledge base from the multiple mapping relationship tables, and adds the corresponding relationship record between the business data field and the knowledge field in the table. The process is based on the complete information determined in the previous processing process, such as the data source name, the source data primary key, the target knowledge base identifier, the document identifier, the segment identifier, and the newly created knowledge field, and strictly follows the preset mapping rule to establish the association between the business data and the knowledge content. The added mapping record contains a complete field corresponding chain to ensure that the subsequent operation can be reversely located to the related content in the knowledge base through the mapping relationship. The update of the mapping relationship table maintains the accurate correspondence between the business data and the knowledge base content, provides a reliable association basis for subsequent data synchronization operation, and records the position information of the newly added knowledge content.

[0110] The technical scheme of the embodiment of the application captures the incremental change event of the business data table in real time through the listening client, generates a data incremental message containing metadata such as operation type, table name and field value, and sends it to the message middleware; the message middleware receives and persistently stores the incremental message to the message queue to ensure the ordered delivery of the message; the synchronization master service obtains the message from the queue, calls the matched synchronization plug-in for processing according to the table name and operation type; the synchronization plug-in analyzes the message content, converts the structured data into standardized knowledge base operation data and feeds back. When processing the addition operation, the synchronization master service obtains the target knowledge segment in the target document, creates the target knowledge field matched with the target data field and constructs the target knowledge data, then constructs the addition instruction containing complete positioning parameters and executes it through the plug-in interface, and finally establishes the field mapping relationship in the mapping relationship table. This new type of search enhanced knowledge base update method realizes the automatic transfer of business data to the knowledge base, greatly improves the timeliness and accuracy of the knowledge base update; especially for the addition update operation, a complete forward synchronization mechanism is established, the standardized field mapping and content conversion process are used to ensure that the business data addition can be accurately transmitted to the knowledge base content, effectively solving the problem of incomplete or non-standard format of knowledge content in the traditional way.

[0111] Embodiment four

[0112] Figure 4 The flowchart of another search enhanced knowledge base update method provided by the fourth embodiment of the application is based on the optimization of the above-mentioned embodiments. The knowledge base operation type includes: deleting data, adding data and modifying data; the modification data operation in the "synchronizing the search enhanced knowledge base matched with the knowledge base operation data through the synchronization master service when receiving the knowledge base operation data" is refined.

[0113] S410, when the business data table is found to have a data increment event, the client generates a data increment message matching the data increment event and sends the data increment message to the message middleware.

[0114] S420, the message middleware stores the real-time received data increment message to the message queue.

[0115] S430, the synchronization master service obtains messages from the message queue in sequence through synchronization, and calls the synchronization plug-in matching the obtained message in each synchronization plug-in for message processing.

[0116] S440, the synchronization plug-in responds to the call of the synchronization master service, parses the message sent by the synchronization master service to obtain knowledge base operation data, and feeds back the knowledge base operation data to the synchronization master service.

[0117] When the knowledge base operation type in the embodiment is modifying data, the knowledge base operation data is to modify the knowledge data matching the target data field in the target document of the target search enhanced knowledge base to the target field value.

[0118] S450, when the received knowledge base operation data is to modify the knowledge data matching the target data field in the target document of the target search enhanced knowledge base to the target field value, the synchronization master service obtains the target mapping relationship table matching the target search enhanced knowledge base from the plurality of mapping relationship tables maintained locally.

[0119] In the embodiment, when the synchronization master service receives the knowledge base operation data of the modification type, it immediately queries the specific mapping relationship table corresponding to the target search enhanced knowledge base from the plurality of mapping relationship tables maintained locally. The step accurately locates the mapping relationship table to be used through the data source name and the target knowledge base identifier. The table records the complete correspondence between the fields of the business data table and the knowledge fields of the knowledge base in detail, including the data source name, the data source primary key, the data set identifier, the document identifier and the segment identifier and other key mapping information. The query process is based on accurate index positioning, which ensures that all mapping records related to the current modification operation can be quickly obtained, providing accurate basis for determining the specific knowledge field and knowledge segment to be modified subsequently.

[0120] S460, the synchronization master service queries the target mapping relationship table, obtains the target knowledge field matching the target data field, and obtains the target knowledge segment to which the target knowledge field belongs.

[0121] In this embodiment, the synchronization master service performs an accurate query based on the acquired target mapping relationship table to determine the specific knowledge content position that needs to be modified. By using the data source name and the source data primary key as query conditions, the target knowledge field identifier that accurately matches the target data field is obtained from the mapping relationship table, and the specific knowledge segment identifier to which the knowledge field belongs is also retrieved. The query process uses the complete field correspondence chain maintained in the mapping relationship table, including dataset identifier, document identifier, and segment identifier, and other multi-level positioning information, to ensure that the specific content position that needs to be updated in the knowledge base can be accurately found. The acquired target knowledge field and segment identifier will be used as key parameters for the construction of subsequent modification instructions, ensuring that changes in business data can be accurately applied to the corresponding field content in the knowledge base, and maintaining the consistency of data and knowledge.

[0122] S470, constructing, by the synchronization master service, a knowledge modification instruction for modifying knowledge data corresponding to the target knowledge field according to the target field value in the target knowledge segment of the target document of the target search enhanced knowledge base.

[0123] In this embodiment, the synchronization master service constructs an accurate knowledge modification instruction based on the acquired target knowledge base identifier, document identifier, segment identifier, and target knowledge field information. The instruction explicitly contains operation type identifier, target positioning parameter, and modification content details, and strictly follows the knowledge base interface specification. The instruction generation process is completely based on the mapping relationship table query result, ensuring that the modification of the business data field can be accurately transmitted to the associated knowledge field content. The modification instruction specifies the specific operation of updating the specified knowledge field content in the determined segment of the specific document of the target knowledge base, including field identifier, new field value, and necessary verification information, providing complete and explicit execution parameters for subsequent execution of the modification operation.

[0124] S480, invoking, by the synchronization master service, a plug-in interface matched with the search enhanced knowledge base to execute the knowledge modification instruction.

[0125] In this embodiment, after the construction of the knowledge modification instruction is completed, the synchronization master service executes the instruction by invoking a plug-in interface matched with the search enhanced knowledge base. The calling process strictly follows the preset interface specification and transmits the operation instruction containing complete modification parameters to the knowledge base processing module. The instruction explicitly specifies the target knowledge base identifier, document identifier, segment identifier, and specific knowledge field that needs to be modified and the updated field value. After receiving the modification request, the knowledge base first verifies the integrity and authority validity of the instruction parameters, and then locates the target knowledge field in the specified knowledge base document segment and updates the field content according to the instruction requirements.

[0126] Further, the entire execution process adopts a transaction mechanism to ensure the atomicity of the operation, that is, the content modification and the related index update are taken as a complete transaction unit, ensuring that the modification operation is either completely successful or completely rolled back. After the execution is completed, the knowledge base returns the operation result state, and the synchronous master service records the operation log and updates the related statistical information according to the result state, ensuring that the modified content can be correctly retrieved and used by the large language model.

[0127] The technical scheme of the embodiment of the application captures the business data table change through the listening client and generates a data increment message, which is transmitted to the synchronous master service through the message middleware; the synchronous master service calls the matching plug-in to convert the message into knowledge base operation data. When the operation type is data modification, the synchronous master service queries the mapping relationship table to obtain the target knowledge field and the segmentation information, and constructs a modification instruction containing the target knowledge base identifier, the document identifier, the segmentation identifier and the target field value; then the plug-in interface is called to execute the instruction, and the specified knowledge field content in the target knowledge segment is accurately updated, ensuring that the business data change is synchronized to the retrieval enhanced knowledge base in real time.

[0128] This new retrieval enhanced knowledge base update method realizes the automatic flow of business data to the retrieval enhanced knowledge base, greatly improving the maintenance efficiency of the retrieval enhanced knowledge base; especially for data modification operation, a precise field-level synchronization mechanism is designed, which realizes bidirectional query of the mapping relationship table and standardized instruction construction, ensuring that the modification of business data can be accurately transmitted to the associated knowledge content, effectively solving the problem of modification asynchronization or omission in the traditional way.

[0129] Embodiment five

[0130] Figure 5 A structure diagram of an update device of a retrieval enhanced knowledge base provided by the fifth embodiment of the application. The device is configured in a knowledge base synchronization system, which includes a listening client, a message middleware, a synchronous master service and a plurality of synchronization plug-ins, as shown in Figure 5 The device includes:

[0131] The listening module 510 is configured to generate a data increment message matched with the data increment event when the listening client listens to the data increment event of the business data table, and send the data increment message to the message middleware;

[0132] The transmission module 520 is configured to store the data increment message received in real time to the message queue through the message middleware;

[0133] The scheduling module 530 is configured to acquire the messages from the message queue through the synchronous master service in sequence, and call the synchronization plug-in matched with the acquired message in each synchronization plug-in for message processing;

[0134] The conversion module 540 is configured to parse the message sent by the synchronization master service to obtain knowledge base operation data in response to a call of the synchronization master service by the synchronization plug-in, and feed back the knowledge base operation data to the synchronization master service.

[0135] The update module 550 is configured to perform data synchronization on the search enhanced knowledge base matched with the knowledge base operation data when the knowledge base operation data is received by the synchronization master service, wherein the search enhanced knowledge base is used when the large language model performs search enhancement generation.

[0136] Further, on the basis of the above-mentioned embodiments, the updating device of the search enhanced knowledge base can further include:

[0137] The registration module is configured to register the plug-in at the synchronization master service according to the business data table processed by the plug-in, the message processing type of the plug-in, and the search enhanced knowledge base required to be synchronized by the plug-in through the synchronization plug-in; wherein the synchronization plug-in that completes the plug-in registration can be called and executed by the synchronization master service.

[0138] On the basis of the above-mentioned embodiments, the conversion module 540 is specifically configured to:

[0139] The target message sent by the synchronization master service is parsed to obtain a plurality of key fields in response to a call of the synchronization master service by the target synchronization plug-in;

[0140] The key fields include a plurality of data in the knowledge base operation type, the target search enhanced knowledge base required to be synchronized, the target document in the target search enhanced knowledge base required to be synchronized, the target data field in the target business data table required to be processed, and the target field value matched with the target data field.

[0141] The obtained plurality of key fields are filled into a preset standardized template to obtain knowledge base operation data matched with the target message.

[0142] On the basis of the above-mentioned embodiments, the knowledge base operation type includes deletion of data, addition of data, and modification of data.

[0143] When the knowledge base operation type is deletion of data, the knowledge base operation data is to delete the knowledge data matched with the target data field in the target document of the target search enhanced knowledge base.

[0144] When the knowledge base operation type is addition of data, the knowledge base operation data is to add the knowledge data matched with the target data field and the target field value in the target document of the target search enhanced knowledge base.

[0145] When the knowledge base operation type is modifying data, the knowledge base operation data is modifying the knowledge data matched with the target data field in the target document of the target search enhanced knowledge base to the target field value.

[0146] On the basis of the above embodiments, the updating module 550 is specifically configured to:

[0147] When the received knowledge base operation data is deleting the knowledge data matched with the target data field in the target document of the target search enhanced knowledge base, the synchronization master service acquires a target mapping relationship table matched with the target search enhanced knowledge base from a plurality of mapping relationship tables maintained locally.

[0148] The mapping relationship table stores the mapping relationship between each data field in the set business data table and each knowledge field in the set search enhanced knowledge base.

[0149] The synchronization master service queries the target mapping relationship table to acquire a target knowledge field matched with the target data field and a target knowledge segment to which the target knowledge field belongs.

[0150] The synchronization master service constructs a knowledge deletion instruction for deleting the knowledge data corresponding to the target knowledge field in the target knowledge segment of the target document of the target search enhanced knowledge base.

[0151] The synchronization master service calls a plug-in interface matched with the search enhanced knowledge base to execute the knowledge deletion instruction.

[0152] The synchronization master service deletes the mapping relationship between the target data field and the target knowledge field in the target mapping relationship table.

[0153] On the basis of the above embodiments, the updating module 550 is specifically configured to:

[0154] When the received knowledge base operation data is adding the knowledge data matched with the target data field and the target field value in the target document of the target search enhanced knowledge base, the synchronization master service acquires a target knowledge segment in the target document.

[0155] The synchronization master service newly creates a target knowledge field matched with the target data field and constructs target knowledge data according to the target knowledge field and the target field value.

[0156] The synchronization master service constructs a knowledge addition instruction for adding the target knowledge data in the target knowledge segment of the target document of the target search enhanced knowledge base.

[0157] The synchronization master service calls a plug-in interface matched with the search enhanced knowledge base to execute the knowledge addition instruction.

[0158] The synchronization master service acquires a target mapping relationship table matched with the target search enhanced knowledge base from a plurality of mapping relationship tables maintained locally, and adds a mapping relationship between the target data field and the target knowledge field in the target mapping relationship table.

[0159] On the basis of the above-mentioned embodiments, the updating module 550 is specifically configured to:

[0160] When the synchronization master service modifies the knowledge data matched with the target data field in the target document of the target search enhanced knowledge base to the target field value according to the received knowledge base operation data, the synchronization master service acquires a target mapping relationship table matched with the target search enhanced knowledge base from a plurality of mapping relationship tables maintained locally;

[0161] The synchronization master service queries the target mapping relationship table to acquire a target knowledge field matched with the target data field, and acquires a target knowledge segment to which the target knowledge field belongs;

[0162] The synchronization master service constructs a knowledge modification instruction for modifying the knowledge data corresponding to the target knowledge field according to the target field value in the target knowledge segment of the target document of the target search enhanced knowledge base;

[0163] The synchronization master service calls a plug-in interface matched with the search enhanced knowledge base to execute the knowledge modification instruction.

[0164] The updating device for a search enhanced knowledge base provided in the embodiments of the present application can execute the updating method for a search enhanced knowledge base provided in any of the embodiments of the present application, and has the function modules and beneficial effects corresponding to the execution method.

[0165] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information comply with relevant laws and regulations and do not violate public order and good customs.

[0166] Embodiment six

[0167] Figure 6 A structural schematic diagram of an electronic device 10 that can be used to implement embodiments of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices such as personal digital assistants, cellular telephones, smartphones, wearable devices (e.g., headsets, glasses, watches, etc.), and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit the implementations of the present application described and / or claimed in this document.

[0168] As shown in Figure 6 The electronic device 10 includes at least one processor 11, and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., connected in communication with the at least one processor 11, wherein the memory stores a computer program executable by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0169] Various components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc., an output unit 17, such as various types of displays, a speaker, etc., a storage unit 18, such as a magnetic disk, an optical disk, etc., and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0170] The processor 11 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs various methods and processes described above, such as a method of retrieving updates of an enhanced knowledge base, performed by a knowledge base synchronization system including a listening client, a message middleware, a synchronization master service, and multiple synchronization plugins, i.e., the method includes:

[0171] generating, by the listening client, a data increment message matched with the data increment event when a data increment event of a business data table is listened to, and sending the data increment message to the message middleware;

[0172] storing, by the message middleware, the data increment message received in real time to a message queue;

[0173] obtaining, by the synchronization master service, messages from the message queue in sequence, and calling a synchronization plugin matched with the obtained message in each synchronization plugin for message processing;

[0174] The synchronization master service sends a message to the synchronization plug-in, and the synchronization plug-in parses the message to obtain knowledge base operation data and feeds back the knowledge base operation data to the synchronization master service.

[0175] The synchronization master service synchronizes data of the search-enhanced knowledge base matched with the knowledge base operation data when receiving the knowledge base operation data, wherein the search-enhanced knowledge base is used when the large language model generates the search-enhanced knowledge base.

[0176] In some embodiments, a method for updating a search-enhanced knowledge base can be implemented as a computer program tangibly embodied in a computer readable storage medium, such as storage unit 18. In some embodiments, portions or all of the computer program can be loaded and / or installed onto electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded onto RAM 13 and executed by processor 11, one or more steps of a method for updating a search-enhanced knowledge base described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform a method for updating a search-enhanced knowledge base by any other suitable means, such as by way of firmware.

[0177] The various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (PLD), a computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0178] Computer programs used to implement the processes of the application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the computer program, when executed by the processor of the machine, implements the functions / acts specified in the flow diagrams and / or block diagrams. The computer program can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0179] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of a machine-readable storage medium will include one or more lines of a program of instructions in a transitory signal, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0180] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0181] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0182] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and VPS service.

[0183] It should be understood that the various forms of flow shown above can be reordered, added to, or have steps deleted. For example, the steps described in the present application can be performed in parallel, in series, or in a different order, as long as the desired results of the technical solutions of the present application can be achieved, which are not limited herein.

[0184] The above detailed description does not constitute a limitation on the protection scope of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for synchronizing a retrieval-enhanced knowledge base, characterized in that, The method is executed by a knowledge base synchronization system, which includes a listening client, a message middleware, a main synchronization service, and multiple synchronization plugins. When the client detects a data increment event in the business data table, it generates a data increment message that matches the data increment event and sends the data increment message to the message middleware. The received incremental data messages are stored in a message queue through a message middleware; The main synchronization service retrieves messages sequentially from the message queue and calls the corresponding synchronization plugin in each synchronization plugin to process the messages. The synchronization plugin responds to the calls of the main synchronization service, parses the messages sent by the main synchronization service to obtain knowledge base operation data, and feeds the knowledge base operation data back to the main synchronization service. When the main service receives knowledge base operation data, it synchronizes the retrieval enhancement knowledge base that matches the knowledge base operation data. The retrieval enhancement knowledge base is used when generating retrieval enhancements in the large language model.

2. The method according to claim 1, characterized in that, Before the main synchronization service calls the synchronization plugin that matches the retrieved message for message processing, the process also includes: Each of the aforementioned synchronization plugins registers itself with the main synchronization service based on the business data table it processes, the message processing type of the plugin, and the retrieval enhancement knowledge base that the plugin needs to synchronize. Among them, the synchronous plugin that has completed plugin registration can be called and executed by the main synchronous service.

3. The method according to claim 1, characterized in that, The synchronization plugin responds to calls from the main synchronization service, parsing the messages sent by the main synchronization service to obtain knowledge base operation data, including: The target synchronization plugin responds to the calls from the main synchronization service, parses the target message sent by the main synchronization service, and obtains several key fields. The key fields include: knowledge base operation type, target retrieval enhancement knowledge base to be synchronized, target document to be synchronized in target retrieval enhancement knowledge base, target data field to be processed in target business data table, and multiple data in target field values ​​that match the target data field. The obtained key fields are filled into a preset standardized template to obtain knowledge base operation data that matches the target message.

4. The method according to claim 3, characterized in that, Knowledge base operations include: deleting data, adding data, and modifying data; When the knowledge base operation type is data deletion, the knowledge base operation data is to delete the knowledge data in the target document of the target retrieval enhancement knowledge base that matches the target data field; When the knowledge base operation type is "add data", the knowledge base operation data is the knowledge data that matches the target data field and the target field value in the target document of the target retrieval enhancement knowledge base. When the knowledge base operation type is "modify data", the knowledge base operation data is to modify the knowledge data in the target document of the target retrieval enhancement knowledge base that matches the target data field to the target field value.

5. The method according to claim 4, characterized in that, When the master service receives knowledge base operation data, it synchronizes the data with the retrieval enhancement knowledge base that matches the knowledge base operation data, including: When the main service receives knowledge base operation data that is used to delete knowledge data that matches the target data field in the target document of the target retrieval enhancement knowledge base, it retrieves the target mapping relationship table that matches the target retrieval enhancement knowledge base from multiple mapping relationship tables maintained locally. The mapping relationship table stores the mapping relationship between each data field in the set business data table and each knowledge field in the set retrieval enhancement knowledge base; By querying the target mapping relationship table through the synchronous main service, the target knowledge field that matches the target data field is obtained, and the target knowledge segment to which the target knowledge field belongs is obtained; By synchronizing the main service, the target knowledge segment of the target document in the target retrieval enhancement knowledge base is built with the target retrieval service, and the knowledge deletion instruction is used to delete the knowledge data corresponding to the target knowledge field. The knowledge deletion instruction is executed by synchronously calling the plugin interface that matches the enhanced knowledge base; By synchronizing the main service, the mapping relationship between the target data field and the target knowledge field is deleted from the target mapping relationship table.

6. The method according to claim 4, characterized in that, When the master service receives knowledge base operation data, it synchronizes the data with the retrieval enhancement knowledge base that matches the knowledge base operation data, including: When the received knowledge base operation data is to add knowledge data that matches the target data field and the target field value to the target document in the target retrieval enhancement knowledge base, the synchronous main service obtains the target knowledge segment in the target document; The target knowledge field is created by synchronizing the main service to match the target data field, and the target knowledge data is constructed based on the target knowledge field and the target field value. Knowledge addition instructions are added to the target knowledge segments of target documents built in the target retrieval enhancement knowledge base through the synchronous main service, adding new target knowledge data. The knowledge addition instruction is executed by synchronously calling the plugin interface that matches the enhanced knowledge base with the main service call; By synchronizing multiple mapping relationship tables maintained locally by the main service, a target mapping relationship table matching the target retrieval enhanced knowledge base is obtained, and the mapping relationship between the target data field and the target knowledge field is added to the target mapping relationship table.

7. The method according to claim 4, characterized in that, When the master service receives knowledge base operation data, it synchronizes the data with the retrieval enhancement knowledge base that matches the knowledge base operation data, including: When the main service receives knowledge base operation data that modifies knowledge data matching the target data field in the target document of the target retrieval enhancement knowledge base to the target field value, it retrieves the target mapping relationship table that matches the target retrieval enhancement knowledge base from multiple mapping relationship tables maintained locally. By querying the target mapping relationship table through the synchronous main service, the target knowledge field that matches the target data field is obtained, and the target knowledge segment to which the target knowledge field belongs is obtained; By synchronizing the main service, the target knowledge segments of the target documents in the target retrieval enhancement knowledge base are built, and knowledge modification instructions are given to modify the knowledge data corresponding to the target knowledge fields according to the target field values. The knowledge modification instructions are executed by synchronously calling the main service and matching the plugin interface with the enhanced knowledge base.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, which enables the at least one processor to perform the synchronization method for the retrieval enhancement knowledge base as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the synchronization method for the retrieval-enhanced knowledge base as described in any one of claims 1-7.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the synchronization method for the retrieval enhancement knowledge base according to any one of claims 1-7.

Images