Method, medium, and apparatus for retrieval enhancement of ai data lakes

Abstract

A retrieval enhancement method, medium and device of an AI data lake relate to the technical field of computers, and the method comprises the following steps: receiving a data processing task, the data processing task being executed based on an agent in an AI data lake; determining a knowledge modality of retrieval enhancement based on the data processing task; determining recall knowledge corresponding to the data processing task from a knowledge base based on the data processing task and the knowledge modality, the recall knowledge being determined through a first candidate list and a second candidate list, the first candidate list and the second candidate list containing knowledge recalled based on different recall modes; and sending the recall knowledge to the agent, so that the agent executes the data processing task based on the recall knowledge. When the agent of the AI data lake executes the data processing task, the required recall knowledge under the knowledge modality can be recalled during the knowledge retrieval enhancement, and the accuracy and availability of the recall knowledge are improved.

Description

Technical Field

[0001] This article relates to the field of computer technology, and more specifically, to a method, medium, and device for enhancing retrieval in an AI data lake. Background Technology

[0002] AI (Artificial Intelligence) data lakes are unified data infrastructures optimized for AI / ML (Machine Learning) and use large model-driven agents to automate data processing tasks. With the widespread application of large language models in code generation, workflow orchestration, and enterprise question answering, agents can reference structured or semi-structured domain knowledge during data processing. Retrieval-Augmented Generation (RAG) involves the agent retrieving several pieces of knowledge most relevant to the current task from a knowledge base before invoking the large language model, using this retrieved knowledge as context to improve the accuracy of the large model's data processing. In related technologies, knowledge retrieval typically involves matching user-input query information with knowledge in the knowledge base, retrieving knowledge with high matching scores. Summary of the Invention

[0003] This content section is provided to briefly introduce the ideas, which will be described in detail in the examples section later. This content section is not intended to identify key or essential features of the claimed content, nor is it intended to limit the scope of the claimed content.

[0004] Firstly, a retrieval enhancement method for AI data lakes is provided, including: Receive data processing tasks, which are executed based on agents in the AI ​​data lake; Based on the data processing task, a knowledge modality for retrieval enhancement is determined. The knowledge modality is used to represent the modality of first knowledge, which is the knowledge that the agent needs to recall for retrieval enhancement when performing the data processing task. Based on the data processing task and the knowledge modality, recall knowledge corresponding to the data processing task is determined from the knowledge base. The knowledge in the knowledge base is associated with modality types. The modality types of the knowledge are used to indicate the modality of applying the knowledge to data processing. The recall knowledge is determined through a first candidate list and a second candidate list. The first candidate list and the second candidate list contain knowledge recalled based on different recall methods. The recall knowledge is sent to the intelligent agent so that the intelligent agent can perform the data processing task based on the recall knowledge.

[0005] In a second aspect, a computer-readable medium is provided having a computer program stored thereon, wherein the computer program, when executed by a processing device, implements the steps of the method described in the first aspect.

[0006] Thirdly, an electronic device is provided, comprising: A storage device on which computer programs are stored; A processing device for executing the computer program in the storage device to implement the steps of the method described in the first aspect.

[0007] Therefore, when an agent in an AI data lake executes a data processing task, modality recognition can be performed first based on the data processing task to determine the knowledge modality for retrieval enhancement. This allows the agent to recall knowledge in the required modality during the knowledge retrieval enhancement process, avoiding the recall of unusable knowledge and preventing the agent from incorrectly calling knowledge and generating unusable data processing task execution results. Furthermore, when determining the recall knowledge, different recall methods can be used to recall and fuse knowledge, improving the accuracy and usability of the recalled knowledge, providing effective data support for the agent to execute data processing tasks, and ensuring the accurate execution of these tasks.

[0008] Other features and advantages will be described in detail in the following examples section. Attached Figure Description

[0009] The above and other features, advantages, and aspects of this document will become more apparent when viewed in conjunction with the accompanying drawings and the following examples. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale. In the drawings: Figure 1 This is an exemplary schematic diagram illustrating an implementation environment.

[0010] Figure 2 This is a flowchart illustrating retrieval enhancement methods for AI data lakes based on certain scenarios.

[0011] Figure 3 This is a block diagram illustrating a retrieval enhancement device for an AI data lake, based on several scenarios.

[0012] Figure 4 These are schematic diagrams of the structure of electronic devices shown in various scenarios. Detailed Implementation

[0013] The following description will be given in more detail with reference to the accompanying drawings. While certain scenarios are shown in the drawings, it should be understood that this document can be implemented in various forms and should not be construed as limited to the scenarios described herein. Rather, these scenarios are provided to provide a more thorough and complete understanding of this document. It should be understood that the accompanying drawings and the scenarios depicted are for illustrative purposes only and are not intended to limit the scope of this document.

[0014] It should be understood that the steps described in the method may be performed in different orders and / or in parallel. Furthermore, the method may include additional steps and / or omit the steps shown. The scope of this document is not limited in this respect.

[0015] The term "comprising" and its variations can be open-ended, meaning "including but not limited to". The term "based on" can mean "at least partially based on". The term "one case" means "at least one case"; the term "another case" means "at least one additional case"; the term "some cases" means "at least some cases". Definitions of other terms will be given in the following description.

[0016] It should be noted that the concepts of "first" and "second" are only used to distinguish different devices, modules or units, and are not used to limit the order of the functions performed by these devices, modules or units or their interdependencies.

[0017] It should be noted that the modifiers “one” and “multiple” can be illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as “one or more”.

[0018] The names of messages or information exchanged between multiple devices are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0019] It is understandable that the data involved in the technical solution (including but not limited to the data itself, the acquisition or use of the data) should comply with the requirements of relevant regulations.

[0020] AI data lakes support the management and access of structured, semi-structured, and unstructured multimodal data, and deeply integrate AI capabilities to achieve intelligent processing of data governance, processing, modeling, and inference throughout the entire process. They serve as high-performance data platforms supporting multimodal AI and large-scale model training. AI data lakes utilize large-model-driven agents to automate data processing tasks. The retrieval enhancement methods for AI data lakes provided in this content can be executed by electronic devices, which can be at least one of terminals or servers. Figure 1 This is an exemplary schematic diagram illustrating an implementation environment; see [link / reference]. Figure 1 The implementation environment includes: terminal 101 and server 102.

[0021] For example, a data platform can be installed on terminal 101, and a data processing task can be triggered on the data platform. The data processing task is sent to server 102, and the task processing result returned by server 102 is received and executed on the platform interface. Server 102 is the backend server of the intelligent analysis agent, used to provide backend services for the data platform, such as parsing data processing tasks, determining the recall knowledge corresponding to the data processing tasks, and executing data processing tasks to process the AI ​​data lake.

[0022] In one scenario, in response to a triggered data processing task for the AI ​​data lake, the data processing task can be sent to server 102 on the platform interface of terminal 101. Server 102 receives and parses the data processing task, determines the recall knowledge corresponding to the task, and sends the recall knowledge to the agent. The agent then executes the data processing task in its execution environment and returns the result to terminal 101. Terminal 101 receives and displays the result on the platform interface. For example, terminal 101 can handle the front-end display while server 102 handles the back-end data processing; alternatively, terminal 101 can handle only part of the data processing. The specific configuration is flexible and not limited to any particular configuration.

[0023] Terminal 101 can be at least one of the following devices: smartphone, smartwatch, desktop computer, laptop, virtual reality terminal, augmented reality terminal, wireless terminal, and laptop computer. Terminal 101 has communication capabilities and can access wired or wireless networks. Terminal 101 can refer to one of multiple terminals, and those skilled in the art will understand that the number of such terminals can be more or less. Server 102 can be an independent physical server, a server cluster composed of multiple physical servers, or a distributed file system. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and artificial intelligence platforms.

[0024] For example, server 102 and terminal 101 are connected directly or indirectly via wired or wireless communication, without limitation. Optionally, the number of servers 102 can be more or less, without limitation. Of course, server 102 may also include other functional servers to provide more comprehensive and diversified services.

[0025] In some cases, retrieval enhancement methods for AI data lakes can be executed via electronic devices, which can be at least one type of terminal device. For example, such as... Figure 2 The diagram shown is a flowchart illustrating retrieval enhancement methods for AI data lakes under various scenarios. Figure 2 As shown, the method may include: In step 21, a data processing task is received, which is executed based on an agent in the AI ​​data lake.

[0026] The data processing tasks include storing, retrieving, and managing the AI ​​data lake; the specific tasks can be determined based on the actual business scenario and are not limited thereto. Users can trigger data processing tasks by inputting descriptive text from the data platform.

[0027] In step 22, based on the data processing task, a knowledge modality for retrieval enhancement is determined. This knowledge modality represents the modality of first knowledge, which is the knowledge that the agent needs to recall for retrieval enhancement when performing the data processing task. The knowledge in the knowledge base can contain multiple modality types to support data processing of different modalities. Therefore, the data processing task can be parsed for modality recognition to determine the modality of the knowledge that the agent needs to recall during retrieval enhancement, i.e., this knowledge modality, so that targeted knowledge recall can be performed based on this knowledge modality during subsequent retrieval enhancement processes.

[0028] In step 23, based on the data processing task and knowledge modality, recall knowledge corresponding to the data processing task is determined from the knowledge base. The knowledge in the knowledge base is associated with modality types, and the modality types of the knowledge are used to indicate the modality of applying the knowledge to data processing. The recall knowledge is determined through a first candidate list and a second candidate list, which contain knowledge recalled based on different recall methods.

[0029] For example, when storing knowledge in a knowledge base, knowledge can be associated with modal types. Modal types can include text, image, video, audio, and multimodal models, etc. Modal types are used as hard constraints to determine whether the knowledge is adopted. For example, knowledge with a text model type can only be adopted when the data processing modality is text; knowledge with an image model type can only be adopted when the data processing modality is image; knowledge with a video model type can only be adopted when the data processing modality is video; knowledge with an audio model type can only be adopted when the data processing modality is audio; when the modal type is multimodal model, it represents that the knowledge is general knowledge and can be adopted in data processing of various modalities.

[0030] For example, when processing data tasks based on relevant technologies, directly retrieving knowledge based on the data processing task might retrieve knowledge of image and video types. However, since the modality corresponding to the data processing task is video, the retrieved image-type knowledge would interfere with the agent's data processing process. Therefore, when retrieving knowledge from the knowledge base, it is possible to combine the data processing task with knowledge modalities enhanced by the data processing task for knowledge retrieval, providing accurate and usable knowledge for subsequent processing by the agent and large models.

[0031] In this way, candidate knowledge of the data processing task in the knowledge modality can be recalled based on different recall methods to obtain a candidate list. In this way, the knowledge in different candidate lists can be fused together to comprehensively determine the recall knowledge corresponding to the retrieval enhancement, thus ensuring the accuracy of the recall knowledge.

[0032] In step 24, the recall knowledge is sent to the agent so that the agent can perform data processing tasks based on the recall knowledge.

[0033] For example, recalled knowledge can be sent to the corresponding agent in the AI ​​data lake. Based on this agent, the recalled knowledge can be concatenated into the prompt word as context. The prompt word can then be used to call the corresponding large language model. The large language model can then perform inference based on the recalled knowledge and the data processing task to obtain the inference result. The inference result can then be displayed or used for data processing operations to achieve the execution of the data processing task.

[0034] When an agent in an AI data lake performs data processing tasks, modality recognition can be performed first based on the data processing task to determine the knowledge modality for retrieval enhancement. This allows the agent to recall knowledge in the required modality during knowledge retrieval enhancement, avoiding the recall of unusable knowledge and preventing the agent from incorrectly calling knowledge and generating unusable data processing task execution results. Furthermore, different recall methods can be used to recall and fuse knowledge during knowledge recall, improving the accuracy and usability of the recalled knowledge and providing effective data support for the agent to perform data processing tasks, ensuring the accurate execution of these tasks.

[0035] In some cases, the knowledge in the knowledge base is indexed through multiple types of storage engines. Each piece of knowledge is represented by a predefined knowledge structure. The index in each storage engine is generated based on the knowledge structure of the knowledge. Different types of storage engines correspond to the same inbound interface. The inbound interface includes a structure generation method and a knowledge inbound method. The structure generation method is used to generate a knowledge structure for each piece of knowledge based on the knowledge document. Different types of storage engines correspond to different implementation methods of the knowledge inbound method.

[0036] For example, in order to adapt the knowledge base to different storage engines, a unified data schema that is independent of specific storage can be predefined, and a unified base class for data entry and retrieval can be abstracted on top of this data schema.

[0037] For example, all backend storage engines can share the same set of field name constants. Field names can include knowledge identifier, name, description, English category, modality type, construction parameters, input, return value, return data type, embedding vector, and keyword field suffixes. Then, a knowledge structure and a retrieval structure can be predefined based on these fields. The knowledge structure contains all the fields included in the knowledge, which can be predefined based on the aforementioned fields. The retrieval structure contains the results retrieved from the knowledge base. It can append score fields and unique knowledge identifier fields to the knowledge structure to normalize the returned results for different types of storage engines.

[0038] The ingestion interface can predefine abstract methods, such as struct generation methods and knowledge ingestion methods. For struct generation methods, these can include common methods such as document loading, text splitting, batch ingestion iteration, and metadata cleaning, with each storage engine corresponding to the same implementation method. For knowledge ingestion methods, since the index types may differ across storage engines, different implementation methods for the knowledge ingestion method are pre-configured for different storage engines to generate the corresponding index for that storage engine.

[0039] For example, the storage engine may include at least two of the following: A general-purpose engine, which can be a common retrieval engine in this field, typically sets the embedding vector to a nearest neighbor vector field when creating the index. For example, an HNSW (Hierarchical Navigable Small World) index might have mapping parameters including the number of nearest neighbor candidates (ef_construction) and the maximum number of adjacent nodes per layer (m). The specific parameter values ​​can be configured based on the actual application scenario. Furthermore, the index also includes both full-text and exact wildcard fields for knowledge identifiers and names.

[0040] Lightweight columnar vector library engines allow you to declare knowledge embedding vectors as fixed-length floating-point arrays when creating an index. When the data volume reaches a threshold, an IVF_PQ (Inverted File-Product Quantization) index is built. The corresponding index construction parameters can include the number of cluster buckets (num_partitions) and the number of vector sub-segments (num_sub_vectors), with specific parameter values ​​set based on the actual application scenario. This type of engine can also include an inverted scalar index to support full-text search, and the dataset can be stored on object storage.

[0041] Managed knowledge base engines can generate JSONL files and upload them to object storage when creating an index, and then trigger the knowledge base import through an SDK (Software Development Kit). The field names of the knowledge are aligned with those of other engines.

[0042] In this way, multiple storage engines of different types can share the same source data of the same knowledge structure, ensuring semantic alignment of the same knowledge across different storage engines. The knowledge embedding vector can be obtained by calling a multimodal vectorization service based on the knowledge. Here, a unified multimodal vectorization service is used for vectorization. For example, the same vectorization service is used when vectorizing knowledge in different types of storage engines to ensure the consistency of the vector space and make the retrieval results in different storage engines comparable.

[0043] Accordingly, determining the recall knowledge corresponding to the data processing task from the knowledge base based on the data processing task and the knowledge modality may include: Based on the data processing task and the knowledge modality, and based on the index in the storage engine of the first type corresponding to the knowledge base, the recalled knowledge corresponding to the data processing task is determined, wherein the first type is determined based on the data processing task.

[0044] The knowledge base provides a unified retrieval portal, allowing for knowledge matching within the knowledge base based on data processing tasks and knowledge modalities. For example, when triggering a data processing task, a user can manually select the storage engine type and pre-configure the retrieval backend field to record the selected type. If the user selects a lightweight columnar vector library engine, the retrieval backend field for the data processing task will have the value "Lightweight Columnar Vector Library Engine," thus obtaining the first type (Lightweight Columnar Vector Library Engine) by parsing the data processing task. Subsequently, based on the data processing task and the knowledge modality, matching is performed from the indexes within the lightweight columnar vector library engine to determine the corresponding recalled knowledge.

[0045] As another example, a default type can be pre-set, and if the user does not select a storage engine, the default type can be used as the first type. In this way, knowledge retrieval based on multiple storage engine types can be supported, and the upper layer of the business can dynamically select among multiple storage engines without being aware of the backend differences of the storage engines.

[0046] For example, the retrieval entry can contain abstract retrieval methods, and different storage engines can implement different methods for these retrieval methods to adapt to the indexes in that storage engine.

[0047] Therefore, by generating a knowledge structure and then using that structure to create indexes for the storage engine, adding a new type of storage engine only requires determining the implementation of a few abstract methods, reducing the need for modifications to the upper-level business logic and supporting flexible switching and selection of storage engines. Furthermore, it allows different types of storage engines to be evaluated against each other using the same annotation set and evaluation criteria, providing data reference for selecting the appropriate storage engine type.

[0048] In some cases, determining the retrieval-enhanced knowledge modality based on the data processing task may include: The description text of the data processing task is matched with a preset modality vocabulary, and the modality matched from the modality vocabulary is taken as the first modality.

[0049] Specifically, a modal vocabulary can be pre-defined based on standardized modal representations. This vocabulary can include modal terms representing images, multimodal expressions, text, documents, large model series, videos, and audio. Correspondingly, modal terms from this vocabulary can be matched against the description text of the data processing task. If a modal term is matched, it is designated as the first modality. For example, if the description text contains "video" and "multimodal expressions," then "video" and "multimodal expressions" can be designated as the first modality.

[0050] The description text of the data processing task is matched with a preset mapping word list, and the modality corresponding to the mapping word matched from the mapping word list is used as the second modality.

[0051] For example, the descriptive text is written in natural language, and its descriptive style varies from person to person. Different users may use different words to describe the same modality. Therefore, a mapping word list can be pre-built based on the synonyms and near-synonyms of modal words. The mapping word list contains the mapping words of modal words and their mappings, such as "image → picture", "visual → picture", "text → text", "image → picture", etc. This mapping word list can be continuously expanded according to business needs and supports mixed Chinese and English input. In this way, if a mapping word in the mapping word list is matched in the descriptive text, the modal word corresponding to that mapping word can be used as the second modality. For example, if the descriptive text contains the mapping word "visual", then the modal word "picture" corresponding to that mapping word can be used as the second modality.

[0052] Then, a knowledge mode is determined based on the first mode and the second mode.

[0053] For example, deduplication can be performed based on the first and second modalities to obtain the knowledge modalities. In the example above, the knowledge modalities determined include images, videos, and multimodalities.

[0054] Therefore, by using the hit retrieval of modality vocabularies and mapping vocabularies, we can quickly realize modality recognition corresponding to data processing tasks, providing hard constraints for subsequent knowledge retrieval, ensuring the accuracy of the retrieved knowledge and the matching degree between the data processing tasks, and improving the accuracy of the processing results of the data processing tasks.

[0055] As another example, the descriptive text of the data processing task can be input into a modality classifier or a large language model so that the model can output knowledge modalities. For example, the modality classifier can be implemented based on a classifier that is common in the field to enhance the synonym / semantic generalization ability.

[0056] In some cases, the recall method includes keyword matching and vector matching. The step of determining the recall knowledge corresponding to the data processing task from the knowledge base based on the data processing task and the knowledge modality may include: The description text of the data processing task is matched with the knowledge in the knowledge base by keywords to obtain a first candidate list, which contains multiple candidate knowledge.

[0057] The keywords of the knowledge in the knowledge base can be used to construct a full-text inverted index. Therefore, matching can be performed based on the full-text inverted index of the descriptive text and the knowledge. The similarity between the keywords in the descriptive text and the full-text inverted index is sorted from high to low, and the top N knowledge in the sorted list are selected as candidate knowledge in the first candidate list. This can be implemented based on the full-text inverted index retrieval method in this field, which will not be elaborated here.

[0058] In some cases, keyword matching is performed between the description text of the data processing task and the knowledge in the knowledge base to obtain a first candidate list, which may include: The description text of the data processing task is matched with each field of the knowledge to obtain the first matching value corresponding to each field. The fields of the knowledge include a modal type field.

[0059] The first matching value can be represented by a Boolean value. Each field of the knowledge can be set based on the actual application scenario, such as including a segmented representation of the knowledge identifier, a non-segmented representation of the knowledge identifier, name, description, modality type, etc. If a field value is matched in the description text, the first matching value of that field is 1; otherwise, the first matching value is 0.

[0060] The knowledge modality is matched with the field value of the modality type field of the knowledge to obtain a second matching value.

[0061] If the determined knowledge modality is not null, it can be matched with the field value of the modality type field of the knowledge. If a field value in the modality type field is matched in the knowledge modality, the second matching value is 1. If no field value in the modality type field is matched in the knowledge modality, the second matching value is 0.

[0062] Based on the first weight corresponding to each first matching value and the second weight corresponding to each second matching value, the first matching value and the second matching value are weighted to obtain the matching score of the knowledge, wherein the second weight is greater than the first weight.

[0063] The first weight corresponding to the first matching value can be the weight of the field corresponding to the first configuration value, such as the weights of the word segmentation representation of the knowledge identifier, the non-word segmentation representation of the knowledge identifier, the name, the description, and the modality type, which are 3, 3, 2, 1, and 1.5 respectively. The second weight corresponding to the second matching value can be preset, and the second weight can be set to be greater than the first weight, such as setting the second weight to 100.

[0064] The matching scores are sorted from highest to lowest, and a first candidate list is determined based on the sorting.

[0065] For example, the knowledge can be sorted from high to low according to the matching score, and the knowledge in the top N of the sorted list can be used as candidate knowledge in the first candidate list.

[0066] Therefore, when performing keyword matching, different weights can be set for different fields corresponding to the hit keywords. In this way, when the modality type of knowledge matches the knowledge modality corresponding to the data processing task, an additional second matching value can be added to the knowledge that matches the knowledge modality to improve the matching score of the knowledge, increase the likelihood of the knowledge being recalled, and improve the modality matching degree between the recalled knowledge and the data processing task.

[0067] The description text of the data processing task is vectorized to obtain the query vector corresponding to the data processing task. For example, a multimodal vectorization service can be invoked based on the description text to obtain the vector corresponding to the description text, which can then be used as the query vector. This can be achieved by using the vectorization service applied when generating knowledge embedding vectors, ensuring the consistency of the vector space.

[0068] Vector matching is performed based on the query vector and the embedding vector of the knowledge in the knowledge base to obtain a second candidate list, which contains multiple candidate knowledge.

[0069] For example, vector matching can be achieved by calculating the cosine similarity between the query vector and each embedded vector. The vectors can be sorted in descending order of cosine similarity, and the knowledge in the top N of the sorted vectors can be used as candidate knowledge in the second candidate list.

[0070] Based on the first candidate list, the second candidate list, and the knowledge modality, the recall knowledge is determined.

[0071] As an example, duplicate knowledge can be removed from the first and second candidate lists to obtain initial knowledge. If the modality type associated with the initial knowledge is the same as the knowledge modality, or if the modality type of the initial knowledge is multimodal, then the initial knowledge is identified as recall knowledge. For example, in the above example, images, videos, and multimodal initial knowledge can be used as recall knowledge. If the modality type of the initial knowledge is text, then the initial knowledge is ignored.

[0072] Therefore, by combining keyword matching and vector matching, the comprehensiveness and accuracy of retrieval enhancement can be improved. At the same time, by combining knowledge modality to determine recall knowledge, the recall knowledge can be filtered based on knowledge modality to determine the recall knowledge that matches the modality of the data processing task, thus providing effective data support for intelligent agents to perform data processing.

[0073] In some cases, determining the recalled knowledge based on the first candidate list, the second candidate list, and the knowledge modality may include: Based on the first candidate list and the second candidate list, a reciprocal ranking fusion is performed to determine the ranking score of each candidate knowledge.

[0074] The ranking score can be calculated using Reciprocal Rank Fusion (RRF). For each candidate knowledge in the candidate list, the reciprocal of the sum of the smoothing coefficient and the ranking rank is used as the ranking score for that candidate knowledge in that candidate list, with the ranking rank starting from 1. Then, the ranking scores of the same candidate knowledge in the first and second candidate lists are summed to obtain the ranking score. As an example, the weights of the first and second candidate lists can be pre-set, and the ranking scores of the same candidate knowledge in the first and second candidate lists are then weighted and summed to obtain the ranking score. RRF does not rely on the comparability of specific similarity scores in the first and second candidate lists, but only on the ranking of knowledge in different candidate lists, thus mitigating the sensitivity of naive weighted fusion to the distribution of similarity scores.

[0075] The candidate knowledge is sorted from highest to lowest score to obtain a first sequence, and initial knowledge is determined based on the first sequence. For example, the first sequence can be obtained by sorting the candidate knowledge from highest to lowest score, and the top M pieces of knowledge in the first sequence can be used as the initial knowledge.

[0076] If the modal type associated with the initial knowledge is the same as the knowledge modality, or if the modal type of the initial knowledge is a multimodal type, then the initial knowledge is identified as recall knowledge.

[0077] As in the example above, images, videos, and multimodal initial knowledge can be used as recall knowledge. If the modal type of the initial knowledge is text, then that initial knowledge is ignored.

[0078] Therefore, while filtering based on knowledge modality, it is possible to retain individual knowledge that matches the knowledge modality. At the same time, it is also possible to retain general knowledge of multimodal types. Multimodal knowledge (such as general model inference scheduling and general file reading and writing) is applicable to all modalities. If hard filtering is performed based on a single query modality, it will reduce the comprehensiveness of the recalled knowledge. Retaining multimodal knowledge will not introduce cross-modal mismatches and will avoid the problem of less recalled knowledge caused by over-filtering, thus ensuring the accuracy and usability of the recalled knowledge.

[0079] In some cases, if the number of recalled knowledge items is less than a preset threshold, some of the recalled knowledge items can be selected from the candidate knowledge items whose associated modal types are different from the knowledge modal. For example, the candidate knowledge items whose associated modal types are different from the knowledge modal can be sorted from high to low according to their ranking scores, and some of the recalled knowledge items can be selected again based on the ranking to ensure the total number of recalled knowledge items.

[0080] As another example, in some cases, if the modality type associated with the candidate knowledge differs from the knowledge modality, a decay coefficient is set. If the decay coefficient is less than 1, the product of the decay coefficient and the ranking score of the candidate knowledge can be used as a new ranking score. The candidate knowledge is then ranked from highest to lowest based on the updated ranking score, and the top H candidate knowledge pieces are used as the recalled knowledge. This allows for filtering of candidate knowledge with ranking scores different from the knowledge modality, reducing the likelihood of this type of knowledge being recalled and improving the accuracy of recalled knowledge.

[0081] In some cases, the method may further include: Based on the data processing task and the recall knowledge, cross-coding is performed to determine the relevance score.

[0082] Among these methods, relevance scores can be determined based on cross-encoders in this field. Cross-encoders can use a joint encoding mechanism to concatenate the description text of the data processing task and the knowledge text of the recalled knowledge into a complete sequence (e.g., ...). <cls>Description text <sep>Knowledge Text <sep>Both texts are fed into the same Transformer model for encoding. Then, a self-attention mechanism allows each word in both texts to perform mutual attention calculations to capture word-level and sentence-level semantic connections and logical dependencies. Finally, the cross-encoder model directly outputs a relevance score between 0 and 1.

[0083] Based on the relevance scores, the knowledge is sorted from high to low to obtain a second sequence of recalled knowledge, thereby achieving accurate sorting of recalled knowledge.

[0084] Accordingly, sending the recalled knowledge to the agent may include: sending the second sequence to the agent, such that the recalled knowledge in the second sequence received by the agent is sorted according to its relevance to the data processing task, so that when the agent calls the large model for inference, it can prioritize the recalled knowledge with high relevance, thereby improving the accuracy of the execution of the data processing task.

[0085] In some cases, the method may further include: Obtain a labeled dataset, wherein each labeled data in the labeled dataset contains a labeled data processing task and the labeled recall knowledge corresponding to the labeled data processing task.

[0086] In this process, pre-annotation can be performed by annotators, such as from user-triggered data processing tasks, to determine the data processing tasks to be annotated, and the recall knowledge corresponding to the data processing tasks to be annotated, with the truly relevant knowledge being used as the annotated recall knowledge.

[0087] Based on the candidate values ​​of the hyperparameters and the labeled data processing task, the test recall knowledge corresponding to each candidate value is determined from the knowledge base. The hyperparameters are used to represent the parameters that constrain the determination of recall knowledge from the knowledge base.

[0088] For example, the hyperparameters can be the number of candidate knowledge N selected, the number of initial knowledge M selected based on the first sequence, and the first weight and the second weight, etc., which can be configured according to the actual application scenario. Taking the number N as an example, the candidate values ​​of the number N can be preset. For example, if the candidate values ​​of N are [2, 20], then with N=2, the test recall knowledge corresponding to the labeled data processing task can be determined from the knowledge base; with N=3, the test recall knowledge corresponding to the labeled data processing task can be determined from the knowledge base; with N=4, the test recall knowledge corresponding to the labeled data processing task can be determined from the knowledge base; and so on, to determine the test recall knowledge under each candidate value of N.

[0089] For each candidate value, an evaluation metric is determined based on the test recall knowledge and the labeled recall knowledge.

[0090] The evaluation metrics can be commonly used retrieval metrics in this field, which can be configured based on actual application scenarios. For example, it can include a hit rate metric, Hit@K, indicating whether the top K test-recalled knowledge items contain labeled recalled knowledge. For instance, the knowledge identifier, name, and description of the test-recalled knowledge can be concatenated into a long string. Each labeled recalled knowledge item is then normalized to remove non-alphanumeric characters. If the long string of the test-recalled knowledge contains a substring of the labeled recalled knowledge, then the test-recalled knowledge is considered to have hit the labeled recalled knowledge. The average hit rate corresponding to the candidate value is determined based on the test-recalled knowledge from each labeled data processing task. For example, the ratio of the number of labeled recalled knowledge items hit by the test-recalled knowledge from the i-th labeled data processing task to the total number of labeled recalled knowledge items from the i-th labeled data processing task can be used as the hit rate of the i-th labeled data processing task, and the average hit rate of all labeled data processing tasks can be used as the average hit rate. Alternatively, the evaluation metric can be Precision@K, representing the precision of the top K test-recalled knowledge items, indicating the proportion of effective knowledge among the top K test-recalled knowledge items. For example, the evaluation metric could also be Recall@K, which is the percentage of standard recalled knowledge that appears in the top K test recalled knowledge.

[0091] The parameter value of the hyperparameter is determined based on the evaluation index of each candidate value of the hyperparameter.

[0092] Subsequently, an evaluation curve can be generated for this hyperparameter, with candidate values ​​on the X-axis and the evaluation metric on the Y-axis. The parameter value of the hyperparameter is then determined based on this evaluation curve. Thus, the parameter value of the hyperparameter can be determined through offline evaluation using labeled data, enabling automatic tuning of the hyperparameter. This makes the iteration of retrieved knowledge results quantifiable, further improving the accuracy of enhanced retrieval.

[0093] Figure 3 The diagram shown is a block diagram of an AI data lake retrieval enhancement device 30 according to some scenarios. The retrieval enhancement device 30 includes: The receiving module 301 is used to receive data processing tasks, which are executed based on agents in the AI ​​data lake; The first determining module 302 is used to determine a knowledge modality for retrieval enhancement based on the data processing task. The knowledge modality is used to represent the modality of first knowledge, which is the knowledge that the agent needs to recall for retrieval enhancement when performing the data processing task. The second determining module 303 is used to determine the recall knowledge corresponding to the data processing task from the knowledge base based on the data processing task and the knowledge modality. The knowledge in the knowledge base is associated with a modality type, and the modality type of the knowledge is used to indicate the modality of applying the knowledge to data processing. The recall knowledge is determined through a first candidate list and a second candidate list. The first candidate list and the second candidate list contain knowledge recalled based on different recall methods. The processing module 304 is used to send the recall knowledge to the intelligent agent, so that the intelligent agent can perform the data processing task based on the recall knowledge.

[0094] Therefore, when an agent in an AI data lake executes a data processing task, modality recognition can be performed first based on the data processing task to determine the knowledge modality for retrieval enhancement. This allows the agent to recall knowledge in the required modality during the knowledge retrieval enhancement process, avoiding the recall of unusable knowledge and preventing the agent from incorrectly calling knowledge and generating unusable data processing task execution results. Furthermore, when determining the recall knowledge, different recall methods can be used to recall and fuse knowledge, improving the accuracy and usability of the recalled knowledge, providing effective data support for the agent to execute data processing tasks, and ensuring the accurate execution of these tasks.

[0095] Optionally, the first determining module 302 includes: The first matching submodule is used to match the description text of the data processing task with a preset modality vocabulary, and to use the modality matched from the modality vocabulary as the first modality. The second matching submodule is used to match the description text of the data processing task with a preset mapping word list, and use the modality corresponding to the mapping word matched from the mapping word list as the second modality; The first determining submodule is used to determine the knowledge mode based on the first mode and the second mode.

[0096] Optionally, the knowledge in the knowledge base is indexed by multiple types of storage engines. Each piece of knowledge is represented by a predefined knowledge structure. The index in each storage engine is generated based on the knowledge structure. Different types of storage engines correspond to the same inbound interface. The inbound interface includes a structure generation method and a knowledge inbound method. The structure generation method is used to generate a knowledge structure for each piece of knowledge based on the knowledge document. Different types of storage engines correspond to different implementation methods of the knowledge inbound method. The second determining module 303 includes: The second determining submodule is used to determine the recall knowledge corresponding to the data processing task based on the data processing task and the knowledge modality, and based on the index in the storage engine of the first type corresponding to the knowledge base, wherein the first type is determined based on the data processing task.

[0097] Optionally, the recall method includes keyword matching and vector matching, and the second determining module 303 includes: The third matching submodule is used to perform keyword matching between the description text of the data processing task and the knowledge in the knowledge base to obtain a first candidate list, which contains multiple candidate knowledge. The first processing submodule is used to vectorize the description text of the data processing task to obtain the query vector corresponding to the data processing task. The fourth matching submodule is used to perform vector matching based on the query vector and the embedding vector of the knowledge in the knowledge base to obtain a second candidate list, which contains multiple candidate knowledge. The third determination submodule is used to determine the recall knowledge based on the first candidate list, the second candidate list, and the knowledge modality.

[0098] Optionally, the third matching submodule is further used for: The description text of the data processing task is matched with each field of the knowledge to obtain the first matching value corresponding to each field. The fields of the knowledge include a modality type field. The knowledge modality is matched with the field value of the modality type field of the knowledge to obtain a second matching value; Based on the first weight corresponding to each first matching value and the second weight corresponding to each second matching value, the first matching value and the second matching value are weighted to obtain the matching score of the knowledge, wherein the second weight is greater than the first weight; The matching scores are sorted from highest to lowest, and a first candidate list is determined based on the sorting.

[0099] Optionally, the third determining submodule is further configured to: Based on the first candidate list and the second candidate list, the inverse ranking fusion is performed to determine the ranking score of each candidate knowledge; The candidate knowledge is sorted from high to low based on its ranking score to obtain a first sequence, and the initial knowledge is determined based on the first sequence. If the modal type associated with the initial knowledge is the same as the knowledge modality, or if the modal type of the initial knowledge is a multimodal type, then the initial knowledge is identified as recall knowledge.

[0100] Optionally, the retrieval enhancement device 30 further includes: The third determining module is used to determine the relevance score by cross-coding based on the data processing task and the recall knowledge; The sorting module is used to sort the relevance scores from high to low to obtain a second sequence of the recalled knowledge. The processing module 304 is used to: send the second sequence to the intelligent agent.

[0101] Optionally, the retrieval enhancement device 30 further includes: The acquisition module is used to acquire the annotation dataset, wherein each annotation data in the annotation dataset contains the annotation data processing task and the annotation recall knowledge corresponding to the annotation data processing task; The fourth determination module is used to determine the test recall knowledge corresponding to each candidate value from the knowledge base based on the candidate values ​​of the hyperparameters and the labeled data processing task. The hyperparameters are used to represent parameters that constrain the determination of recall knowledge from the knowledge base. The fifth determination module is used to determine the evaluation index for each candidate value based on the test recall knowledge and the labeled recall knowledge; The sixth determining module is used to determine the parameter value of the hyperparameter based on the evaluation index of each candidate value of the hyperparameter.

[0102] The following is for reference. Figure 4 It shows an electronic device suitable for implementing the above method (e.g. Figure 1 The diagram below shows the structure of the terminal device or server 600. The terminal device may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Personal Computers), PMPs (Portable Media Players), in-vehicle terminals (such as in-vehicle navigation terminals), and fixed terminals such as digital TVs (Televisions), desktop computers, etc. Figure 4 The electronic device shown is merely an example and should not be construed as limiting its functionality or scope of use.

[0103] like Figure 4 As shown, electronic device 600 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 602 or a program loaded from storage device 608 into random access memory (RAM) 603. The random access memory 603 also stores various programs and data required for the operation of electronic device 600. The processing unit 601, ROM 602, and RAM 603 are interconnected via bus 604. An input / output (I / O) interface 605 is also connected to bus 604.

[0104] Typically, the following devices can be connected to the input / output interface 605: input devices 606 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 607 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 608 including, for example, magnetic tape, hard disk, etc.; and communication devices 609. Communication device 609 allows electronic device 600 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 4 An electronic device 600 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0105] In particular, depending on certain circumstances, the processes described in the flowchart above can be implemented as computer software programs. For example, a computer program product is provided, comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowchart. This computer program can be downloaded and installed from a network via communication device 609, or installed from storage device 608, or installed from read-only memory 602. When the computer program is executed by processing device 601, it performs the functions defined in the above-described methods.

[0106] It should be noted that the aforementioned computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM, or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In one case, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In another case, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. The transmitted data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (Radio Frequency), etc., or any suitable combination thereof.

[0107] In some scenarios, clients and servers can communicate using any currently known or future-developed network protocol, such as HTTP (Hypertext Transfer Protocol), and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include Local Area Networks (LANs), Wide Area Networks (WANs), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., ad-hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0108] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0109] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: receive a data processing task, the data processing task being executed based on an agent in an AI data lake; determine a retrieval enhancement knowledge modality based on the data processing task, the knowledge modality representing a modality of first knowledge, the first knowledge being the knowledge required by the agent to retrieve for retrieval enhancement when executing the data processing task; determine, based on the data processing task and the knowledge modality, retrieve knowledge corresponding to the data processing task from a knowledge base, the knowledge in the knowledge base being associated with a modality type, the modality type of the knowledge indicating the modality of applying the knowledge for data processing, the retrieved knowledge being determined through a first candidate list and a second candidate list, the first candidate list and the second candidate list containing knowledge retrieved based on different retrieval methods; and send the retrieved knowledge to the agent, causing the agent to execute the data processing task based on the retrieved knowledge.

[0110] Computer program code for performing the above operations can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include, but are not limited to, object-oriented programming languages, as well as conventional procedural programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0111] The flowcharts and block diagrams in the accompanying figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products under various scenarios. Each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that, in some alternative cases, the functions indicated in the blocks may occur in a different order than those indicated in the figures. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0112] The modules mentioned above can be implemented in software or hardware. In some cases, the name of a module does not necessarily limit the module itself; for example, a receiving module can also be described as "a module that receives and processes data tasks."

[0113] The functions described above can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field-Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application-Specific Standard Parts (ASSPs), Systems on Chips (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0114] In this context, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0115] The above description is merely illustrative and explains the technical principles employed. Those skilled in the art should understand that the scope of this document is not limited to the specific combinations of the above-described technical features, but should also cover any combination of the above-described technical features or their equivalents without departing from the above concept.

[0116] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. Multitasking and parallel processing may be advantageous in certain contexts. Similarly, while some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of this paper. Certain features described in the context of a single example can also be implemented in combination in a single example. Conversely, various features described in the context of a single example can also be implemented individually or in any suitable sub-combination in multiple examples.

[0117] Although this document has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims. Regarding the aforementioned apparatus, the specific manner in which the various modules perform their operations has already been described in detail in the section concerning the method, and will not be elaborated upon here.< / sep> < / sep> < / cls>

Claims

1. A retrieval enhancement method for AI data lakes, comprising: Receive data processing tasks, which are executed based on agents in the AI ​​data lake; Based on the data processing task, a knowledge modality for retrieval enhancement is determined. The knowledge modality is used to represent the modality of first knowledge, which is the knowledge that the agent needs to recall for retrieval enhancement when performing the data processing task. Based on the data processing task and the knowledge modality, recall knowledge corresponding to the data processing task is determined from the knowledge base. The knowledge in the knowledge base is associated with modality types. The modality types of the knowledge are used to indicate the modality of applying the knowledge to data processing. The recall knowledge is determined through a first candidate list and a second candidate list. The first candidate list and the second candidate list contain knowledge recalled based on different recall methods. The recall knowledge is sent to the intelligent agent so that the intelligent agent can perform the data processing task based on the recall knowledge.

2. The method according to claim 1, wherein determining the retrieval-enhanced knowledge modality based on the data processing task includes: The description text of the data processing task is matched with a preset modality vocabulary, and the modality matched from the modality vocabulary is taken as the first modality. The description text of the data processing task is matched with a preset mapping word list, and the modality corresponding to the mapping word matched from the mapping word list is used as the second modality; The knowledge mode is determined based on the first mode and the second mode.

3. The method according to claim 1, wherein the knowledge in the knowledge base is indexed by multiple types of storage engines, each piece of knowledge is represented by a predefined knowledge structure, the index in each storage engine is generated based on the knowledge structure, different types of storage engines correspond to the same inbound interface, the inbound interface includes a structure generation method and a knowledge inbound method, the structure generation method is used to generate a knowledge structure for each piece of knowledge based on a knowledge document, and different types of storage engines correspond to different implementation methods of the knowledge inbound method; The step of determining the recall knowledge corresponding to the data processing task from the knowledge base based on the data processing task and the knowledge modality includes: Based on the data processing task and the knowledge modality, and based on the index in the storage engine of the first type corresponding to the knowledge base, the recalled knowledge corresponding to the data processing task is determined, wherein the first type is determined based on the data processing task.

4. The method according to claim 1, wherein the recall method includes keyword matching and vector matching, and the step of determining the recall knowledge corresponding to the data processing task from the knowledge base based on the data processing task and the knowledge modality includes: The description text of the data processing task is matched with the knowledge in the knowledge base by keywords to obtain a first candidate list, which contains multiple candidate knowledge. The description text of the data processing task is vectorized to obtain the query vector corresponding to the data processing task. Vector matching is performed based on the query vector and the embedding vector of the knowledge in the knowledge base to obtain a second candidate list, which contains multiple candidate knowledge. Based on the first candidate list, the second candidate list, and the knowledge modality, the recall knowledge is determined.

5. The method according to claim 4, wherein keyword matching is performed between the description text of the data processing task and the knowledge in the knowledge base to obtain a first candidate list, comprising: The description text of the data processing task is matched with each field of the knowledge to obtain the first matching value corresponding to each field. The fields of the knowledge include a modality type field. The knowledge modality is matched with the field value of the modality type field of the knowledge to obtain a second matching value; Based on the first weight corresponding to each first matching value and the second weight corresponding to each second matching value, the first matching value and the second matching value are weighted to obtain the matching score of the knowledge, wherein the second weight is greater than the first weight; The matching scores are sorted from highest to lowest, and a first candidate list is determined based on the sorting.

6. The method according to claim 4, wherein determining the recalled knowledge based on the first candidate list, the second candidate list, and the knowledge modality comprises: Based on the first candidate list and the second candidate list, the inverse ranking fusion is performed to determine the ranking score of each candidate knowledge; The candidate knowledge is sorted from high to low based on its ranking score to obtain a first sequence, and the initial knowledge is determined based on the first sequence. If the modal type associated with the initial knowledge is the same as the knowledge modality, or if the modal type of the initial knowledge is a multimodal type, then the initial knowledge is identified as recall knowledge.

7. The method according to any one of claims 1-6, further comprising: Cross-coding is performed based on the data processing task and the recall knowledge to determine the relevance score; Based on the relevance scores, the second sequence of recalled knowledge is obtained by sorting the knowledge from high to low. Sending the recall knowledge to the agent includes: sending the second sequence to the agent.

8. The method according to any one of claims 1-6, further comprising: Obtain a labeled dataset, wherein each labeled data in the labeled dataset contains a labeled data processing task and the labeled recall knowledge corresponding to the labeled data processing task; Based on the candidate values ​​of the hyperparameters and the labeled data processing task, the test recall knowledge corresponding to each candidate value is determined from the knowledge base. The hyperparameters are used to represent parameters that constrain the determination of recall knowledge from the knowledge base. For each candidate value, an evaluation metric is determined based on the test recall knowledge and the labeled recall knowledge; The parameter value of the hyperparameter is determined based on the evaluation index of each candidate value of the hyperparameter.

9. A computer-readable medium having a computer program stored thereon, wherein, When executed by a processing device, the computer program implements the steps of the method described in any one of claims 1-8.

10. An electronic device, comprising: A storage device on which computer programs are stored; A processing device for executing the computer program in the storage device to implement the steps of the method according to any one of claims 1-8.

Images