Knowledge migration method and system supporting multi-modal task
By generating the optimal transfer strategy through cross-modal alignment and policy optimization, the problems of low efficiency and unstable performance in multimodal task transfer are solved, and efficient and flexible knowledge transfer and model optimization are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIHANG UNIV
- Filing Date
- 2026-02-04
- Publication Date
- 2026-05-15
AI Technical Summary
Existing technologies are inefficient and unstable in multimodal and heterogeneous task migration, especially in low-resource scenarios. Traditional methods require complex fine-tuning or rely on a large amount of labeled data, resulting in excessively long migration time and insufficient knowledge adaptation.
By acquiring source task data and target task requirements, we extract source feature knowledge sets and perform cross-modal alignment, construct standardized knowledge modules, use policy optimization agents to generate optimal transfer strategies, and optimize the target task model through local fine-tuning and evaluation index systems.
It achieves efficient knowledge transfer for multimodal tasks, enhances policy generalization ability, ensures a balance between performance and efficiency, and adapts to the needs of different target tasks.
Smart Images
Figure CN122047422A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence task transfer technology, specifically to a knowledge transfer method and system that supports multimodal tasks. Background Technology
[0002] In applications such as natural language processing, computer vision, and robot control, achieving cross-task and cross-modal knowledge reuse and model transfer is a crucial step in artificial intelligence task transfer technology.
[0003] Current mainstream knowledge transfer methods mainly include two categories: parameter transfer and feature transfer. Parameter transfer methods typically use the model parameters (such as neural network weights) trained on the source task as the initial parameters of the target task model, and then fine-tune them. Feature transfer methods, on the other hand, focus on using feature representations learned in the source task (such as word vectors, image embeddings, etc.), and introduce them into the target task model through feature alignment, feature distillation, and other means to enhance the representational ability of the target model.
[0004] However, when dealing with multimodal and heterogeneous task migration, the above methods need to introduce complex fine-tuning mechanisms or rely on a large amount of labeled data in order to achieve high performance, resulting in excessively long migration time; while in low-resource scenarios, the lack of knowledge adaptation leads to large performance fluctuations. Summary of the Invention
[0005] (a) Technical problems to be solved To address the shortcomings of existing technologies, this invention provides a knowledge transfer method and system that supports multimodal tasks, solving the problem of efficiency and performance imbalance in traditional knowledge transfer methods.
[0006] (II) Technical Solution To achieve the above objectives, the present invention provides the following technical solution: In a first aspect, the present invention provides a knowledge transfer method supporting multimodal tasks, the method comprising: S1, obtain the source task data and target task requirements input by the user; the source task data includes the source task model and source task description set according to the source task, and the target task requirements include the target task type, data volume, preset performance threshold and resource limit; S2, based on the target task requirements, extract the source feature knowledge set of the source task data and perform cross-modal alignment to obtain standardized knowledge modules; parse the target task requirements to obtain target task features; S3. Based on the current policy library and the target task features, obtain an initial knowledge transfer policy, input the initial knowledge transfer policy into a pre-built policy optimization agent, and output the optimal knowledge transfer policy; select a matching base model from the source task model according to the target task features. S4. Based on the optimal knowledge transfer strategy and the standardized knowledge module, perform local fine-tuning of the basic model for knowledge transfer to obtain the initial target task model; S5, the initial target task model is tested and updated through a pre-built evaluation index system to obtain the target task model.
[0007] Preferably, the strategy optimization agent selects the transfer strategy with the highest reward value as the optimal knowledge transfer strategy; wherein, the formula for calculating the reward value is: in, R As a reward value, Loss The target task loss value, T The migration time is represented by α and β, which are weighting coefficients.
[0008] Preferably, the step of extracting the source feature knowledge set from the source task data based on the target task requirements and performing cross-modal alignment to obtain a standardized knowledge module includes: Knowledge is extracted from the source task model using a hierarchical knowledge extraction algorithm to obtain a source feature knowledge set; wherein, the source feature knowledge set includes basic feature knowledge, logical reasoning knowledge, and task decision knowledge; Construct a knowledge graph based on the source feature knowledge set; If the source task and the target task are of different modalities, the knowledge graph is mapped to a unified vector space through a contrastive learning algorithm to obtain standardized knowledge modules.
[0009] Preferably, obtaining the initial knowledge transfer strategy based on the current strategy library and the characteristics of the target task includes: Call the basic strategy template in the current strategy library that has the highest matching degree with the target task features; An initial knowledge transfer strategy is obtained based on the standardized knowledge module and the basic strategy template; wherein, the initial knowledge transfer strategy includes the knowledge reuse ratio, the number of fine-tuning iterations, and the model parameter update method.
[0010] Preferably, the step of detecting and updating the initial target task model through a pre-constructed evaluation index system includes: If the performance indicators do not meet the standards, return to step S2 and re-optimize the node association accuracy of the knowledge graph; When the efficiency index fails to meet the standard, return to step S3 to simplify the local fine-tuning process; If the stability metric fails to meet the standard, return to step S2 to optimize the robustness of the cross-modal alignment.
[0011] Preferably, the method further includes: The reusable knowledge modules extracted from the standardized knowledge modules by the optimal knowledge transfer strategy are updated to the knowledge graph; the optimal knowledge transfer strategy is updated to the strategy library.
[0012] Preferably, the target task features are obtained by parsing the target task requirements through a task feature matching algorithm; the target task features include input / output features, data distribution features, and core task objectives.
[0013] Secondly, the present invention also provides a knowledge transfer system supporting multimodal tasks, the system comprising: The knowledge transfer module is used to perform the following steps: S1, obtain the source task data and target task requirements input by the user; the source task data includes the source task model and source task description set according to the source task, and the target task requirements include the target task type, data volume, preset performance threshold and resource limit; S2, based on the target task requirements, extract the source feature knowledge set of the source task data and perform cross-modal alignment to obtain standardized knowledge modules; parse the target task requirements to obtain target task features; S3. Based on the current policy library and the target task features, obtain an initial knowledge transfer policy, input the initial knowledge transfer policy into a pre-built policy optimization agent, and output the optimal knowledge transfer policy; select a matching base model from the source task model according to the target task features. S4. Based on the optimal knowledge transfer strategy and the standardized knowledge module, perform local fine-tuning of the basic model for knowledge transfer to obtain the initial target task model; S5, The initial target task model is detected and updated through a pre-built evaluation index system to obtain the target task model; The processing module is used to process the target task model obtained through the knowledge transfer module to meet the target task requirements and obtain the target result.
[0014] Thirdly, the present invention also provides a computer-readable storage medium storing a computer program for supporting knowledge transfer in multimodal tasks, wherein the computer program causes a computer to execute the knowledge transfer method for supporting multimodal tasks as described above.
[0015] Fourthly, the present invention also provides an electronic device, comprising: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the programs including a knowledge transfer method for performing any of the above-described multimodal tasks.
[0016] (III) Beneficial Effects This invention provides a knowledge transfer method and system that supports multimodal tasks. Compared with existing technologies, it has the following advantages: 1. This application achieves systematic extraction and structured reorganization of source task knowledge by extracting source feature knowledge sets from source task data that correspond to the requirements of the target task, and performing cross-modal alignment in the case of cross-modal processing to integrate them into standardized knowledge modules, thereby avoiding knowledge fragmentation.
[0017] 2. This application matches the initial knowledge transfer strategy with a local policy library based on the characteristics of the target task and optimizes the intelligent agent optimization strategy with a preset intelligent agent strategy to obtain the optimal knowledge transfer strategy. The transfer strategy generation mechanism constructed in this way can dynamically adapt to different target tasks, which greatly improves the strategy generalization ability.
[0018] 3. This application achieves real-time monitoring of the migration process through a pre-constructed evaluation index system, and realizes iterative optimization of the model by updating the model through the detection results. Attached Figure Description
[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0020] Figure 1 This is a flowchart illustrating the knowledge transfer method provided in an embodiment of the present invention. Detailed Implementation
[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are described clearly and completely. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0022] This application provides a knowledge transfer method and system that supports multimodal tasks, solving the problems of efficiency and performance imbalance caused by knowledge fragmentation in traditional knowledge transfer methods, as well as poor generalization and adaptability when facing multimodal tasks, thereby achieving efficient knowledge transfer.
[0023] The technical solution in this application is to solve the above-mentioned technical problems, and the general idea is as follows: Traditional knowledge transfer methods for young learners have three major problems: (1) Knowledge fragmentation: The knowledge of the source task (such as features, rules, and parameters) is not systematically reorganized, which leads to the break of knowledge association during the transfer process, which can easily cause "negative transfer" and reduce the performance of the target task; (2) Strategy rigidity: Existing transfer strategies are mostly manually designed or fixed for specific tasks, lacking the ability to adapt to different task types (such as classification, generation, and detection), with poor generalization and high adjustment costs; (3) Efficiency and performance imbalance: In pursuit of high performance, complex fine-tuning mechanisms are introduced or a large amount of labeled data is relied upon, resulting in excessively long transfer time; while in low-resource scenarios, performance fluctuates greatly due to insufficient knowledge adaptation.
[0024] To address the above problems, this invention provides a knowledge transfer method and system that supports multimodal tasks. The objectives are: (1) to achieve systematic extraction and structured reorganization of source task knowledge to avoid knowledge fragmentation; (2) to construct a transfer strategy generation mechanism that can dynamically adapt to different target tasks to improve the generalization ability of the strategy; and (3) to significantly improve transfer efficiency while ensuring transfer performance, thereby achieving a balance between performance and efficiency.
[0025] To better understand the above technical solution, a detailed description of the technical solution will be provided below in conjunction with the accompanying drawings and specific implementation methods. The implementation method of this embodiment uses a computer device as the core execution entity, and executes it once through a built-in software module. The steps are closely linked to form a complete task migration process.
[0026] like Figure 1 As shown in the figure, this application provides a knowledge transfer method that supports multimodal tasks, including: S1, obtain the source task data and target task requirements input by the user; the source task data includes the source task model and source task description set according to the source task, and the target task requirements include the target task type, data volume, preset performance threshold and resource limitations. S2, based on the target task requirements, extract the source feature knowledge set of the source task data and perform cross-modal alignment to obtain standardized knowledge modules; parse the target task requirements to obtain target task features; S3: Based on the current policy library and the characteristics of the target task, obtain the initial knowledge transfer policy, input the initial knowledge transfer policy into the pre-built policy optimization agent, and output the optimal knowledge transfer policy; select the matching base model from the source task model according to the characteristics of the target task. S4. Based on the optimal knowledge transfer strategy and standardized knowledge modules, the basic model is locally fine-tuned for knowledge transfer to obtain the initial target task model.
[0027] S5 uses a pre-built evaluation index system to test and update the initial target task model, thus obtaining the target task model.
[0028] The knowledge transfer method provided in this embodiment extracts standardized knowledge modules from the source task, then obtains the optimal knowledge transfer strategy through agent interaction. The optimal strategy is used to transfer reusable knowledge modules from the standardized knowledge modules to the initial task model, resulting in the target task model. The construction of standardized knowledge modules achieves systematic extraction and structured reorganization of source task knowledge, avoiding knowledge fragmentation. The optimal knowledge transfer strategy mechanism, derived through agent interaction for strategy optimization, can dynamically adapt to knowledge transfer for different target tasks, greatly improving the strategy's generalization ability.
[0029] In one embodiment, step S1 involves obtaining source task data and target task requirements input by the user; the source task data includes a source task model and a source task description set according to the source task, and the target task requirements include the target task type, data volume, preset performance threshold, and resource limitations.
[0030] In this embodiment, the input interface module of the computer device receives the source task data and target task requirements input by the user. The source task data includes the source task model (including model parameters, training dataset, and model structure file) and the source task description (such as task type, core objective, and application scenario). The target task requirements include key information such as task type, data volume, preset performance threshold (such as accuracy ≥ 85%), and resource limitations (such as migration time ≤ 2 hours).
[0031] The data acquisition performed in this embodiment provides basic data support for subsequent knowledge extraction and strategy generation, ensuring that the framework clearly defines the starting point and goal of the migration.
[0032] In one embodiment, step S2 involves extracting the source feature knowledge set from the source task data based on the target task requirements and performing cross-modal alignment to obtain a standardized knowledge module; then parsing the target task requirements to obtain the target task features. The specific implementation of obtaining the standardized knowledge module in this embodiment includes the following steps: Step S21: Extract knowledge from the source task model using a hierarchical knowledge extraction algorithm to obtain a source feature knowledge set; wherein, the source feature knowledge set includes basic feature knowledge, logical reasoning knowledge, and task decision knowledge.
[0033] Step S22: Construct a knowledge graph based on the source feature knowledge set.
[0034] Step S23: If the source task and the target task are different modalities, the knowledge graph is mapped to a unified vector space through a contrastive learning algorithm to obtain standardized knowledge modules.
[0035] In this embodiment, knowledge extraction and reconstruction are performed using the knowledge processing module of the computer device. Specifically, a hierarchical knowledge extraction algorithm is employed to extract basic feature knowledge (such as word vector features of text and pixel texture features of images) from the input layer of the source task model, logical reasoning knowledge (such as the association weights between features and decision paths) from the hidden layer, and task decision knowledge (such as category determination rules for classification tasks and prediction formulas for regression tasks) from the output layer. Based on the extracted three types of knowledge units, a task knowledge graph is constructed according to the structure of "concept-relationship-rule". Knowledge units are used as "concept nodes", and the relationships between knowledge units (such as the correspondence between features and decision rules) are used as "relation edges", forming a standardized knowledge template.
[0036] In this embodiment, alignment is performed using a cross-modal knowledge alignment module. If the source task and the target task are of different modalities (e.g., text to image), the cross-modal knowledge alignment module uses a contrastive learning algorithm to map the knowledge of different modalities to a unified vector space. The calculation formula is as follows: in, x For text modal knowledge vectors, y For image modality knowledge vectors, Enc() is a pre-trained cross-modal encoder (a known quantity, pre-trained and stored by the framework), and Sim(x,y) is the vector similarity (an unknown quantity, used to measure the alignment effect). As an example rather than a limitation, alignment is considered complete when Sim(x,y)≥0.8.
[0037] In this embodiment, the target task requirements are analyzed using a task feature matching algorithm to obtain the target task features. These target task features include input / output features (such as the sequence output features of a text generation task and the bounding box prediction features of an object detection task), data distribution features (such as data sparsity and class balance), and core task objectives (such as the fluency of generated text and the accuracy of detection results). It is important to clarify that input and output features are generally described together. For large-scale model-related tasks, there is usually an input before there is an output. For example, the input to a text-to-image task is a detailed text, and the output is the corresponding generated image; the input to an image recognition task is an image, and the output is a text.
[0038] The hierarchical knowledge extraction, knowledge graph construction, and cross-modal knowledge alignment provided in this embodiment break the fragmented state of source task knowledge, provide structured reorganization to form reusable knowledge modules, solve the problem of fine-tuning the knowledge association break in traditional methods, and lay the foundation for subsequent strategy adaptation.
[0039] In one embodiment, step S3 involves obtaining an initial knowledge transfer strategy based on the current policy library and the characteristics of the target task, inputting the initial knowledge transfer strategy into a pre-built policy optimization agent, and outputting the optimal knowledge transfer strategy; then, selecting a matching base model from the source task model according to the characteristics of the target task. The specific implementation of this embodiment includes the following steps: Step S31: Call the basic strategy template in the current strategy library that has the highest matching degree with the target task features.
[0040] Step S32: Obtain an initial knowledge transfer strategy based on standardized knowledge modules and basic strategy templates; wherein, the initial knowledge transfer strategy includes the knowledge reuse ratio, the number of fine-tuning iterations, and the model parameter update method.
[0041] In this embodiment, a pre-defined policy library built into the computer device is used, and the basic policy template with the highest matching degree to the target task features is called. For example, and not limited to, low-resource tasks correspond to the "knowledge-guided fine-tuning template," and cross-domain tasks correspond to the "global policy reconstruction template." The global policy reconstruction template is used to reconstruct the policy when the target task and the source task are identified as belonging to different modalities. Cross-task adaptation optimization: If it is a different task within the same domain (e.g., sentiment analysis to text summarization), the local policy fine-tuning submodule is triggered, adjusting only parameters directly related to the task type. If it is a cross-domain task (e.g., image classification to object detection), the global policy reconstruction submodule is triggered, reconfiguring the knowledge reuse path and model training process, i.e., repeating steps S1-S5 to obtain a new target task model.
[0042] By employing a knowledge transfer strategy, standardized knowledge templates are filled into the knowledge reuse parameters of the basic strategy template to obtain reusable knowledge modules. The initial strategy includes key configurations such as the knowledge reuse ratio (e.g., 60% of the source task decision knowledge is reused), the number of fine-tuning iterations (e.g., 50 rounds), and the model parameter update method (e.g., freezing the underlying parameters and fine-tuning the top-level parameters).
[0043] The task feature matching and initial strategy generation in this embodiment generate reusable knowledge modules from standardized knowledge modules, achieving initial adaptation between the strategy and the target task. This avoids the rigid limitations of traditional fixed strategies and improves the targeting of strategy generation.
[0044] In this embodiment, the initial transfer strategy is input into a pre-built policy optimization agent, and the target task loss value (such as the cross-entropy loss of a classification task) and transfer efficiency (such as the improvement in model performance per unit time) are used as reward functions to build a reinforcement learning feedback environment.
[0045] The agent iterates and tries out different scenarios in the environment, calculating the reward value after each round of adjustments, as shown in the following formula: Where R is the reward value (unknown), Loss is the target task loss value (known, calculated from the model validation set), T is the transfer time (known, obtained from the computer device's timing module), and α (value 0.6) and β (value 0.4) are weight coefficients (known, based on extensive experimental presets). When R reaches its maximum value, the optimal transfer strategy is output, at which point parameters such as the knowledge reuse ratio and the number of fine-tuning iterations are optimally configured.
[0046] In this embodiment, the optimal migration strategy is output with the goal of maximizing the reward value through interactive training between the agent and the environment. The migration strategy parameters are dynamically optimized to ensure that the strategy meets both performance and efficiency requirements and improves the adaptability of the strategy to different tasks.
[0047] In one embodiment, step S4 involves performing local fine-tuning of the basic model based on the optimal knowledge transfer strategy and standardized knowledge modules to obtain the initial target task model.
[0048] In this embodiment, based on the optimal transfer strategy, standardized knowledge modules are injected into the base model of the target task (such as the pre-trained language model GPT-2 or the image model ResNet), and local fine-tuning is performed according to the number of iterations and parameter update method configured by the strategy to generate the initial target task model.
[0049] This embodiment translates the reorganized knowledge and optimized strategies into a directly applicable model, which is the core execution step for achieving task migration.
[0050] In one embodiment, step S5 involves detecting and updating the initial target task model using a pre-built evaluation index system to obtain the target task model. The specific implementation of this step includes the following steps: If the performance metrics do not meet the standard, return to step S2 to re-optimize the node association accuracy of the knowledge graph; if the efficiency metrics do not meet the standard, return to step S3 to simplify the local fine-tuning process; if the stability metrics do not meet the standard, return to step S2 to optimize the robustness of cross-modal alignment.
[0051] Update the reusable knowledge modules extracted from the standardized knowledge modules by the optimal knowledge transfer strategy to the knowledge graph; update the optimal knowledge transfer strategy to the strategy library.
[0052] In this embodiment, a preset evaluation index system is used to test the initial target task model, including performance indicators (such as accuracy and F1 score for classification tasks; BLEU score and perplexity PPL for generation tasks), efficiency indicators (total migration time and GPU resource utilization), and stability indicators (performance fluctuation range under different data volumes).
[0053] The evaluation results are compared with preset thresholds in the target task requirements. If the thresholds are not met, closed-loop optimization is triggered: when performance indicators are not met, feedback is sent to step S2 to re-optimize the node association accuracy of the knowledge graph; when efficiency indicators are not met, feedback is sent to step S3 to simplify the strategy process (e.g., reduce the number of fine-tuning iterations); when stability indicators are not met, feedback is sent to step S2 to enhance the robustness of cross-modal alignment. Steps S2 to S5 are repeated until all indicators meet the preset thresholds.
[0054] This embodiment addresses the problem of traditional methods that "emphasize process and neglect feedback" by monitoring migration effects in real time and then performing closed-loop feedback to correct process defects.
[0055] Output the optimized target task model (including model file and instructions for use); at the same time, update the target task knowledge (such as sequence generation knowledge and detection and localization knowledge) generated during this migration process to the task knowledge graph, add the optimal migration strategy to the strategy library, and complete the incremental learning and updating of the framework.
[0056] This embodiment delivers the migration results while accumulating knowledge and strategy resources to improve the efficiency of subsequent framework migrations.
[0057] As an example, and not a limitation, in text classification task transfer, the source task is sentiment analysis, and the target task is news classification. This framework extracts semantic features and classification rules from sentiment analysis, constructs a knowledge graph, generates a transfer strategy adapted to news classification, and ultimately achieves improved accuracy and reduced transfer time with low resources.
[0058] In cross-modal transfer from image to text, the source task is image classification, and the target task is image description generation. A cross-modal alignment module maps visual features and textual semantics to a unified space, generating a transfer strategy that integrates visual and linguistic knowledge, significantly improving the fluency and relevance of the generated text.
[0059] The embodiments of this application can be adapted to more than 5 types of tasks, such as text classification, generation, image classification, object detection, and robot planning. It has high generalization and reduces the cost of strategy adaptation.
[0060] This invention also provides a knowledge transfer system supporting multimodal tasks, comprising: The knowledge transfer module is used to perform the following steps: S1, obtain the source task data and target task requirements input by the user; the source task data includes the source task model and source task description set according to the source task, and the target task requirements include the target task type, data volume, preset performance threshold and resource limitations. S2, based on the target task requirements, extract the source feature knowledge set of the source task data and perform cross-modal alignment to obtain standardized knowledge modules; parse the target task requirements to obtain target task features; S3: Based on the current policy library and the characteristics of the target task, obtain the initial knowledge transfer policy, input the initial knowledge transfer policy into the pre-built policy optimization agent, and output the optimal knowledge transfer policy; select the matching base model from the source task model according to the characteristics of the target task. S4. Based on the optimal knowledge transfer strategy and standardized knowledge modules, the basic model is locally fine-tuned for knowledge transfer to obtain the initial target task model.
[0061] S5, the initial target task model is tested and updated through a pre-built evaluation index system to obtain the target task model; The processing module is used to process the target task model obtained through the knowledge transfer module to meet the target task requirements and obtain the target result.
[0062] It is understood that the knowledge transfer system supporting multimodal tasks provided in this embodiment of the invention corresponds to the knowledge transfer method supporting multimodal tasks described above. The explanations, examples, and beneficial effects of the relevant content can be referred to the corresponding content in the knowledge transfer method supporting multimodal tasks, and will not be repeated here.
[0063] This invention also provides a computer-readable storage medium storing a computer program for supporting a knowledge transfer method for multimodal tasks, wherein the computer program causes a computer to execute the knowledge transfer method for multimodal tasks as described above.
[0064] This application also provides an electronic device, including: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the programs including a knowledge transfer method for performing a multimodal task as described above.
[0065] In summary, compared with existing technologies, it has the following beneficial effects: 1. The knowledge transfer method provided in this embodiment of the invention extracts standardized knowledge modules from the source task, obtains the optimal knowledge transfer strategy through agent interaction, and transfers reusable knowledge modules from the standardized knowledge modules to the initial task model using the optimal knowledge transfer strategy to obtain the target task model. The construction of standardized knowledge modules realizes the systematic extraction and structured reorganization of source task knowledge, avoiding knowledge fragmentation. The optimal knowledge transfer strategy mechanism obtained through the interaction of the strategy-optimized agent can dynamically adapt to the knowledge transfer of different target tasks, greatly improving the strategy generalization ability.
[0066] 2. The hierarchical knowledge extraction, knowledge graph construction, and cross-modal knowledge alignment provided by the embodiments of the present invention break the fragmented state of source task knowledge, provide structured reorganization to form reusable knowledge modules, and solve the problem of fine-tuning the knowledge association break in traditional methods.
[0067] 3. The embodiments of the present invention generate reusable knowledge modules from standardized knowledge modules, realizing the initial adaptation of strategies and target tasks, avoiding the rigid limitations of traditional fixed strategies, and improving the pertinence of strategy generation.
[0068] 4. In this embodiment of the invention, the agent is trained through interaction with the environment, and the optimal transfer strategy is output with the goal of maximizing the reward value. The transfer strategy parameters are dynamically optimized to ensure that the strategy meets both performance and efficiency requirements and improves the adaptability of the strategy to different tasks.
[0069] 5. The embodiments of the present invention solve the problem of "emphasizing process and neglecting feedback" in traditional methods by monitoring the migration effect in real time and then performing closed-loop feedback to correct process defects.
[0070] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0071] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A knowledge transfer method supporting multimodal tasks, characterized in that, The method includes: S1, obtain the source task data and target task requirements input by the user; the source task data includes the source task model and source task description set according to the source task, and the target task requirements include the target task type, data volume, preset performance threshold and resource limit; S2, based on the target task requirements, extract the source feature knowledge set of the source task data and perform cross-modal alignment to obtain standardized knowledge modules; parse the target task requirements to obtain target task features; S3. Based on the current policy library and the target task features, obtain an initial knowledge transfer policy, input the initial knowledge transfer policy into a pre-built policy optimization agent, and output the optimal knowledge transfer policy; select a matching base model from the source task model according to the target task features. S4. Based on the optimal knowledge transfer strategy and the standardized knowledge module, perform local fine-tuning of the basic model for knowledge transfer to obtain the initial target task model; S5, the initial target task model is tested and updated through a pre-built evaluation index system to obtain the target task model.
2. The knowledge transfer method according to claim 1, characterized in that, The strategy optimization agent selects the transfer strategy with the highest reward value as the optimal knowledge transfer strategy; wherein, the formula for calculating the reward value is: in, R As a reward value, Loss The target task loss value, T The migration time is represented by α and β, which are weighting coefficients.
3. The knowledge transfer method according to claim 1, characterized in that, The process of extracting the source feature knowledge set from the source task data based on the target task requirements and performing cross-modal alignment to obtain a standardized knowledge module includes: Knowledge is extracted from the source task model using a hierarchical knowledge extraction algorithm to obtain a source feature knowledge set; wherein, the source feature knowledge set includes basic feature knowledge, logical reasoning knowledge, and task decision knowledge; Construct a knowledge graph based on the source feature knowledge set; If the source task and the target task are of different modalities, the knowledge graph is mapped to a unified vector space through a contrastive learning algorithm to obtain standardized knowledge modules.
4. The knowledge transfer method according to claim 1, characterized in that, The initial knowledge transfer strategy obtained based on the current strategy library and the characteristics of the target task includes: Call the basic strategy template in the current strategy library that has the highest matching degree with the target task features; An initial knowledge transfer strategy is obtained based on the standardized knowledge module and the basic strategy template; wherein, the initial knowledge transfer strategy includes the knowledge reuse ratio, the number of fine-tuning iterations, and the model parameter update method.
5. The knowledge transfer method according to claim 3, characterized in that, The step of detecting and updating the initial target task model through a pre-built evaluation index system includes: If the performance indicators do not meet the standards, return to step S2 and re-optimize the node association accuracy of the knowledge graph; When the efficiency index fails to meet the standard, return to step S3 to simplify the local fine-tuning process; If the stability metric fails to meet the standard, return to step S2 to optimize the robustness of the cross-modal alignment.
6. The knowledge transfer method according to claim 3, characterized in that, The method further includes: The reusable knowledge modules extracted from the standardized knowledge modules by the optimal knowledge transfer strategy are updated to the knowledge graph; the optimal knowledge transfer strategy is updated to the strategy library.
7. The knowledge transfer method according to claim 1, characterized in that, The target task features are obtained by parsing the target task requirements using a task feature matching algorithm; the target task features include input and output features, data distribution features, and core task objectives.
8. A knowledge transfer system supporting multimodal tasks, characterized in that, The system includes: The knowledge transfer module is used to perform the following steps: S1, obtain the source task data and target task requirements input by the user; the source task data includes the source task model and source task description set according to the source task, and the target task requirements include the target task type, data volume, preset performance threshold and resource limit; S2, based on the target task requirements, extract the source feature knowledge set of the source task data and perform cross-modal alignment to obtain standardized knowledge modules; parse the target task requirements to obtain target task features; S3. Based on the current policy library and the target task features, obtain an initial knowledge transfer policy, input the initial knowledge transfer policy into a pre-built policy optimization agent, and output the optimal knowledge transfer policy; select a matching base model from the source task model according to the target task features. S4. Based on the optimal knowledge transfer strategy and the standardized knowledge module, perform local fine-tuning of the basic model for knowledge transfer to obtain the initial target task model; S5, The initial target task model is detected and updated through a pre-built evaluation index system to obtain the target task model; The processing module is used to process the target task model obtained through the knowledge transfer module to meet the target task requirements and obtain the target result.
9. A computer-readable storage medium, characterized in that, It stores a computer program for supporting knowledge transfer in multimodal tasks, wherein the computer program causes a computer to perform the knowledge transfer method for supporting multimodal tasks as described in any one of claims 1 to 7.
10. An electronic device, characterized in that, include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the programs including a knowledge transfer method for supporting multimodal tasks as described in any one of claims 1 to 7.