Image editing task processing method and device based on hybrid expert model, equipment and medium
By analyzing the hierarchical expert model structure and task complexity and combining it with the preset memory library to determine the expert activation sequence, the problem of computational resource waste in the hybrid expert model is solved, and efficient processing of complex image editing tasks is achieved.
Patent Information
- Application Number
- CN202510870820.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-26
- Publication Date
- 2025-09-23
AI Technical Summary
Existing hybrid expert models need to activate a large number of expert models or design extremely large models when processing complex tasks, resulting in a sharp increase in computing resource consumption and making it difficult to cope with changing task requirements.
A layered expert model structure is adopted, and a task processing strategy of serial processing between different layers and parallel processing within the same layer is utilized. The initial expert model is created through task complexity analysis, and the expert activation sequence is determined according to the preset memory library to achieve division of labor and cooperation among expert models.
It effectively avoids the waste of computing resources, improves the adaptability of the expert model to the task, ensures the consistency of the processing results, and can adapt to complex and changing task requirements.
Smart Images
Figure CN120689469A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, device, equipment and medium for processing image editing tasks based on a hybrid expert model. Background Art
[0002] A hybrid expert model is a deep learning architecture designed to improve overall performance by combining the predictions of multiple expert models. The core idea is to assign input data to different expert sub-models and then combine the outputs of all sub-models to produce the final result. Each expert model specializes in processing a specific type of data, thereby improving the model's efficiency and accuracy.
[0003] Currently, the method of using a mixture of experts model to process tasks is to utilize a single level of expert division of labor. When handling complex tasks, a single-level MoE (Mixture of Experts) architecture usually requires activating a large number of experts or designing an extremely large expert model, resulting in a sharp increase in computing resource consumption and difficulty in coping with complex and changing task requirements. Summary of the Invention
[0004] In view of this, the present invention aims to provide a method, apparatus, device, and medium for processing image editing tasks based on a hybrid expert model. By setting up a layered expert model structure and utilizing a task processing strategy of serial processing between different layers and parallel processing within the same layer, this method achieves division of labor and cooperation among the expert models, avoiding the problem of a dramatic increase in computing resources caused by using a single-layer expert model architecture. The specific solution is as follows:
[0005] In a first aspect, the present application provides a method for processing image editing tasks based on a hybrid expert model, comprising:
[0006] Obtaining a target image editing task, and performing complexity analysis on the target image editing task to obtain a task complexity corresponding to the target image editing task;
[0007] Creating a target number of initial expert models at each level of a target task processing system according to the complexity of the task; wherein the target task processing system includes a macro layer, an intermediate layer, and a micro layer;
[0008] Determining a target expert activation sequence corresponding to the target image editing task based on a similarity between the target image editing task and historical image editing tasks in a preset memory library; wherein the preset memory library is used to store the historical expert activation sequence corresponding to the historical image editing task, and the target expert activation sequence includes an expert combination and an expert activation order corresponding to the target image editing task;
[0009] According to the target expert activation sequence, a target expert model is determined from the trained expert models corresponding to each of the initial expert models, the target expert model is activated based on the expert activation sequence, and the target image editing task is processed using each of the target expert models in different layers; wherein, the target expert models in the same layer adopt a parallel processing strategy, and the target expert models between different layers adopt a serial processing strategy.
[0010] Optionally, after creating a target number of initial expert models at each level of the target task processing system according to the task complexity, the method further includes:
[0011] Training each of the initial expert models using a first target number of training samples to obtain the trained expert models corresponding to each of the initial expert models;
[0012] The initial router in the target task processing system is trained using a second target number of training samples to obtain a corresponding target router; wherein the target router is used to determine a target expert activation sequence corresponding to the target image editing task.
[0013] Optionally, before determining the target expert activation sequence corresponding to the target image editing task based on the similarity between the target image editing task and historical image editing tasks in a preset memory library, the method further includes:
[0014] Obtaining the historical image editing tasks, and obtaining the historical feature vectors corresponding to each of the historical image editing tasks, the historical expert activation sequences, and the historical task processing scores corresponding to each of the historical expert activation sequences;
[0015] The preset memory library is constructed using the historical feature vectors, the historical expert activation sequences, and the historical task processing scores.
[0016] Optionally, the processing of the target image editing task by using the target expert models in different layers includes:
[0017] Analyzing the target image editing task based on each target expert model in the macro layer to determine a target processing strategy corresponding to the target image editing task;
[0018] The target processing strategy and each target expert model in the intermediate layer are used to perform feature transformation on the target image editing task to obtain a corresponding intermediate result, and the intermediate result is adjusted using each target expert model in the micro layer to process the target image editing task.
[0019] Optionally, determining a target expert activation sequence corresponding to the target image editing task based on similarity between the target image editing task and historical image editing tasks in a preset memory library includes:
[0020] Determining whether the similarity between the target feature vector of the target image editing task and the historical feature vectors of each of the historical image editing tasks in the preset memory library is less than a preset similarity threshold;
[0021] If the similarity between the target feature vector and any historical feature vector is not less than the preset similarity threshold, then determining whether the historical task processing score corresponding to any historical feature vector is lower than the preset score threshold; if the historical task processing score corresponding to any historical feature vector is not lower than the preset score threshold, then determining the expert activation sequence corresponding to any historical feature vector as the target expert activation sequence;
[0022] If the similarity between the target feature vector and any historical feature vector is less than the preset similarity threshold, the target expert activation sequence corresponding to the target image editing task is generated by using the target routing.
[0023] Optionally, after processing the target image editing task using the target expert models in different layers, the method further includes:
[0024] Scoring the target expert activation sequence based on the processing results of the target image editing task by each target expert model to obtain a target task processing score corresponding to the target expert sequence;
[0025] Determine whether the target task processing score is lower than the preset score threshold. If the target task processing score is not lower than the preset score threshold, store the target feature vector, the target expert activation sequence and the target task processing score corresponding to the target image editing task into the preset memory library to update the preset memory library.
[0026] Optionally, the image editing task processing method based on the hybrid expert model further includes:
[0027] Determine the task feature information of the target image editing task, and fine-tune each of the target expert models corresponding to the target image editing task according to the task feature information to obtain a corresponding fine-tuned expert model, and control each of the fine-tuned expert models to collaboratively process the target image editing task.
[0028] In a second aspect, the present application provides an image editing task processing device based on a hybrid expert model, comprising:
[0029] A complexity analysis module is used to obtain a target image editing task and perform complexity analysis on the target image editing task to obtain a task complexity corresponding to the target image editing task;
[0030] an expert model creation module, configured to create a target number of initial expert models at each level of a target task processing system according to the complexity of the task; wherein the target task processing system includes a macro level, an intermediate level, and a micro level;
[0031] an activation sequence determination module, configured to determine a target expert activation sequence corresponding to the target image editing task based on a similarity between the target image editing task and historical image editing tasks in a preset memory library; wherein the target expert activation sequence includes an expert combination and an expert activation order corresponding to the target image editing task;
[0032] A task processing module is used to determine the target expert model from the trained expert models corresponding to each of the initial expert models according to the target expert activation sequence, activate the target expert model based on the expert activation sequence, and use each of the target expert models in different layers to process the target image editing task; wherein, the target expert models in the same layer adopt a parallel processing strategy, and the target expert models between different layers adopt a serial processing strategy.
[0033] In a third aspect, the present application provides an electronic device, comprising:
[0034] Memory, used to store computer programs;
[0035] A processor is used to execute the computer program to implement the aforementioned image editing task processing method based on the hybrid expert model.
[0036] In a fourth aspect, the present application provides a computer-readable storage medium for storing a computer program, which, when executed by a processor, implements the aforementioned image editing task processing method based on a hybrid expert model.
[0037] The present application first obtains a target image editing task and performs a complexity analysis on the target image editing task to obtain the task complexity corresponding to the target image editing task, and then creates a target number of initial expert models at each level of the target task processing system according to the task complexity; wherein the target task processing system includes a macro layer, an intermediate layer and a micro layer, and then determines a target expert activation sequence corresponding to the target image editing task based on the similarity between the target image editing task and the historical image editing tasks in a preset memory library; wherein the preset memory library is used to store the historical expert activation sequence corresponding to the historical image editing task, and the target expert activation sequence includes the expert combination and expert activation sequence corresponding to the target image editing task; finally, according to the target expert activation sequence, a target expert model is determined from the trained expert models corresponding to each of the initial expert models, the target expert model is activated based on the expert activation sequence, and the target image editing task is processed using each of the target expert models in different layers; wherein the target expert models in the same layer adopt a parallel processing strategy, and the target expert models between different layers adopt a serial processing strategy. It can be seen that this application realizes the division of labor and cooperation of expert models by setting up a layered expert model structure and utilizing the task processing strategy of serial processing between different layers and parallel processing within the same layer, thereby avoiding the problem of a sharp increase in computing resources caused by adopting a single-level expert model architecture; by creating expert models according to the complexity of the tasks, the adaptability of the expert models to the tasks is improved, and the problem of waste of computing resources is avoided; by determining the expert activation sequence corresponding to the image editing task and using the adapted expert model for task processing, the consistency between the expert model for task processing and the image editing task is guaranteed, so that this solution can adapt to and handle complex and changing task requirements. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.
[0039] Figure 1 This is a flow chart of a method for processing image editing tasks based on a hybrid expert model disclosed in this application;
[0040] Figure 2 This is a flowchart of a specific method for processing image editing tasks based on a hybrid expert model disclosed in this application;
[0041] Figure 3 This is a flowchart of an image editing task processing disclosed in this application;
[0042] Figure 4 This is a schematic diagram of the structure of an image editing task processing device based on a hybrid expert model disclosed in this application;
[0043] Figure 5 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION
[0044] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.
[0045] Currently, methods for processing tasks using hybrid expert models require activating a large number of experts or designing extremely large expert models, resulting in a sharp increase in computing resource consumption and making it difficult to cope with complex and changing task requirements. To this end, this application provides an image editing task processing method based on a hybrid expert model. By setting up a layered expert model structure and utilizing a task processing strategy of serial processing between different layers and parallel processing within the same layer, this method achieves division of labor and cooperation among expert models, avoiding the problem of a sharp increase in computing resources caused by using a single-layer expert model architecture.
[0046] See also Figure 1 As shown, the embodiment of the present invention discloses a method for processing image editing tasks based on a hybrid expert model, comprising:
[0047] Step S11: Acquire a target image editing task, and perform complexity analysis on the target image editing task to obtain the task complexity corresponding to the target image editing task.
[0048] This embodiment discloses a method for processing image editing tasks based on a hybrid expert model. Through a macro, intermediate, and micro three-layer expert structure and a dynamic activation mode of hierarchical serial and intra-layer parallel, combined with LoRA parameter efficient technology and memory mechanism, it achieves efficient and accurate processing of complex tasks. The specific process is as follows Figure 2 As shown, including:
[0049] Step 1. Initialize the system: Initialize the pre-trained basic model. The task complexity evaluation module creates N1, N2, and N3 LoRA expert adapters for the three levels (macro, meso, and micro) respectively, and the parameters of each expert are randomly initialized.
[0050] Step 2: Pre-training of expert capabilities: Using a diverse task dataset, randomly assign each layer to activate experts and train their basic capabilities to ensure that they have the basic ability to handle tasks in their area of expertise.
[0051] Step 3, router training: Freeze the expert parameters and train the dynamic router so that it can accurately select the most suitable expert combination based on the input features and task type to minimize the task processing loss.
[0052] Step 4: Joint fine-tuning: Unfreeze the expert parameters, train the router and expert parameters simultaneously, optimize the overall system performance, and establish a collaborative working mode among experts.
[0053] Step 5, memory library construction: Use the validation set data to record the task characteristics, the best expert activation mode and the processing result score, and build an initial memory library to provide a reference for fast routing.
[0054] Step 6: Memory optimization training: The training system uses the memory library for similarity matching and expert selection, while optimizing the memory update mechanism to balance memory reuse and exploration learning.
[0055] Step 7: Task adaptability optimization: Use domain-specific tasks for targeted fine-tuning to improve the system's performance in the target application scenario while maintaining general processing capabilities.
[0056] Step 8. Multi-level collaboration optimization: Conduct special training for complex tasks that require cross-level collaboration, strengthen information transmission and decision-making chains between levels, and improve overall collaboration efficiency.
[0057] This embodiment designs a task complexity evaluation module to calculate the task complexity index by analyzing the distribution characteristics, degree of variation, and structural complexity of the input data (i.e., the target image editing task). This index is used to determine the number of experts activated at each layer and the allocation of computing resources.
[0058] Step S12: creating a target number of initial expert models at each level of the target task processing system according to the task complexity; wherein the target task processing system includes a macro layer, an intermediate layer, and a micro layer.
[0059] It can be understood that, in this embodiment, the newly created routing system does not have the ability to select the expert model corresponding to the image editing task based on information such as the complexity of the task and the task characteristics, and the newly created expert model does not have the ability to process the image editing task. Therefore, in this embodiment, after creating a target number of initial expert models at each level of the target task processing system according to the task complexity, it also includes: using a first target number of training samples to train each initial expert model to obtain the trained expert model corresponding to each of the initial expert models; using a second target number of training samples to train the initial routers in the target task processing system to obtain the corresponding target routers; wherein the target routers are used to determine the target expert activation sequence corresponding to the target image editing task.
[0060] Specifically, the process of pre-training the expert model is as follows: using 50K diverse content creation samples (i.e., the first target number of training samples), randomly assigning activation experts to each layer, and training the basic capabilities of each expert. For example, the "style transfer" experts in the middle layer focus on processing style transfer-related samples, and the "object generation" experts process samples with new elements added.
[0061] It should be noted that this implementation uses the parameter-efficient LoRA technology to implement each expert model, significantly reducing the number of parameters and computing resource requirements: the system uses a shared pre-trained base model at the bottom layer, which remains frozen during the inference process and does not perform parameter updates. Each expert model is essentially a specific low-rank adapter (LoRA) for the base model, including a low-rank decomposition matrix and , where r is much smaller than min(d,k) and is usually set to 1% to 5% of the parameter dimension of the corresponding layer of the base model. The form is combined with the basic model parameters W to form a parameter matrix ,in is a scaling factor used to adjust the expert influence.
[0062] The router training process includes freezing the expert parameters, training the dynamic router with 30,000 samples (the second target number of training samples), enabling it to analyze the features of the input text and reference images, and accurately select the most suitable expert combination. For example, when identifying the task of "adding objects while maintaining the original style," the macro-level "content planning" and "style judgment" experts, the intermediate-level "object generation" and "style transfer" experts, and the micro-level "consistency check" and "texture optimization" experts are activated.
[0063] In addition, this implementation also requires joint fine-tuning of the router and the initial expert model: unfreezing the expert parameters of each initial expert model, using 20K high-quality samples to simultaneously train the router and expert parameters, optimizing the overall system performance, ensuring that the experts activated in parallel within the layer can work together, and outputting the fusion results to the next layer of experts for processing.
[0064] This embodiment can avoid the problem of wasting computing resources by creating too many expert models at the same time by pre-creating a preset number of expert models according to the task complexity corresponding to the input image editing task.
[0065] Step S13: Determine a target expert activation sequence corresponding to the target image editing task based on the similarity between the target image editing task and the historical image editing tasks in a preset memory library; wherein the preset memory library is used to store the historical expert activation sequence corresponding to the historical image editing task, and the target expert activation sequence includes the expert combination and expert activation order corresponding to the target image editing task.
[0066] In this embodiment, before determining the target expert activation sequence corresponding to the target image editing task based on the similarity between the target image editing task and the historical image editing tasks in the preset memory library, it also includes: obtaining historical image editing tasks, and obtaining historical feature vectors, historical expert activation sequences and historical task processing scores corresponding to each historical image editing task; using historical feature vectors, historical expert activation sequences and historical task processing scores to construct a preset memory library; specifically, using 5K verification set data, recording the feature vectors (historical feature vectors), hierarchical serial expert activation paths and success scores of each creative task (i.e., historical image editing tasks), and constructing an initial memory library, for example, recording the optimal expert activation sequence for the "photo to comic style" task: first activating the macro-layer "style judgment" + "task understanding", then activating the meso-layer "style conversion" + "color adjustment", and finally activating the micro-layer "texture optimization" + "artistic enhancement".
[0067] This embodiment designs a dynamic intelligent routing mechanism that adaptively selects the optimal expert combination based on input features and task complexity. The routing system (target router) includes the following key components:
[0068] Feature Extraction and Task Analyzer: This component is responsible for extracting key features from input data and generating a task representation vector. This component utilizes a lightweight yet efficient feature extraction network to capture essential features from multimodal input. The task representation vector contains information such as task type, complexity, and key areas, providing a basis for subsequent expert selection.
[0069] Expert Matching and Selector: Based on the task representation vector, the system calculates the fitness score of each expert and selects the most suitable expert combination. This selection process utilizes a weighted probability selection strategy, balancing determinism and exploration to avoid expert selection being trapped in local optima. The system supports multi-level linkage selection. The selection results of macro-level experts influence the activation probability distribution of mid-level and micro-level experts, ensuring the synergy of expert combinations. When processing new tasks, the system automatically adjusts the exploration rate to increase the probability of selecting underutilized experts, thereby expanding the system's capabilities.
[0070] A structured memory database is maintained to store historical successful cases, including information such as input feature vectors, task types, expert activation patterns, and result scores. The database utilizes a hierarchical index structure to support efficient similarity retrieval. When receiving a new task, the system first calculates the similarity between the input and historical cases in the database. If a high-similarity match is found, the corresponding expert activation pattern is prioritized, avoiding repeated routing calculations and improving processing efficiency.
[0071] Accordingly, in this embodiment, the process of determining the target expert activation sequence corresponding to the target image editing task based on the similarity between the target image editing task and the historical image editing tasks in the preset memory library may specifically include:
[0072] Determine whether the similarity between the target feature vector of the target image editing task and the historical feature vectors of each historical image editing task in the preset memory library is less than a preset similarity threshold; if the similarity between the target feature vector and any historical feature vector is not less than the preset similarity threshold, then determine whether the historical task processing score corresponding to any historical feature vector is lower than the preset score threshold; if the historical task processing score corresponding to any historical feature vector is not lower than the preset score threshold, then determine the expert activation sequence corresponding to any historical feature vector as the target expert activation sequence; if the similarity between the target feature vector and any historical feature vector is less than the preset similarity threshold, then use the target routing to generate the target expert activation sequence corresponding to the target image editing task.
[0073] Furthermore, this implementation employs a sparse activation strategy, activating some or all experts at each level to participate in the computation, significantly reducing computational complexity. Activated experts are assigned weight coefficients via a gating network, enabling weighted fusion of expert outputs. The system also incorporates a knowledge-sharing mechanism among experts, allowing them to share portions of the low-rank representation space, further improving parameter efficiency while ensuring professional differentiation. For tasks of varying complexity, the task complexity assessment module adaptively adjusts the number of activated experts. Simple tasks may only activate a small number of experts, while complex tasks require the collaborative activation of a larger number of experts.
[0074] In summary, this embodiment reuses successful expert activation sequences through similarity calculation. For example, when the similarity between a new task and the "photo to watercolor" task in the memory library reaches a preset similarity threshold (e.g., 0.85), the expert-level activation sequence for that task (the expert activation sequence) is directly used as the initial configuration. If the similarity between the current image editing task and any historical image editing tasks in the preset memory library does not exceed the preset similarity threshold, the trained target router is used to generate the expert activation sequence corresponding to the current image editing task. By initially matching from the preset memory library, the acquisition of expert activation sequences is accelerated while avoiding wasted computing resources.
[0075] This embodiment uses a hierarchical, parallel activation model to select and activate experts. First, the top-K macro-level experts are activated in parallel (typically K = 2, for example, to activate both "task understanding" and "style judgment" experts simultaneously). After the macro-level processing completes, corresponding mid-level experts are activated in parallel based on their outputs. Finally, micro-level experts are activated to complete the overall processing flow. The number of experts activated at each level can be dynamically adjusted based on task complexity. Simple tasks may require only one expert per level, while complex tasks may require two macro-level experts, three to four mid-level experts, and two to three micro-level experts.
[0076] In summary, in addition to selecting and creating an expert model based on task complexity, this embodiment also comprehensively considers factors such as the task type, task characteristics, and the similarity between the target image editing task and historical image editing tasks in a preset memory library to select an expert model corresponding to the image editing task. By selecting an expert model based on factors such as the task type, task characteristics, and task complexity corresponding to the input image editing task, the matching between the expert model and the image editing task is ensured, thereby improving the reliability of the task processing results and avoiding the problem of activating too many expert models and wasting computing resources.
[0077] Step S14: determine the target expert model from the trained expert models corresponding to each of the initial expert models according to the target expert activation sequence, activate the target expert model based on the expert activation sequence, and use the target expert models in different layers to process the target image editing task; wherein, the target expert models in the same layer adopt a parallel processing strategy, and the target expert models between different layers adopt a serial processing strategy.
[0078] In this embodiment, when processing the target image editing task, the expert models at different layers are responsible for different tasks, and the experts between the layers rely on each other to jointly complete the processing of the target image editing task. In addition, this embodiment can also fine-tune the expert model according to the task feature information of the task to improve the reliability of the expert model in processing the task.
[0079] In this embodiment, after the target image editing task is processed using the target expert models in different layers, it also includes: scoring the target expert activation sequence based on the processing results of the target image editing task by each target expert model to obtain the target task processing score corresponding to the target expert sequence; judging whether the target task processing score is lower than a preset scoring threshold; if the target task processing score is not lower than the preset scoring threshold, storing the target feature vector, target expert activation sequence and target task processing score corresponding to the target image editing task to a preset memory library to update the preset memory library.
[0080] Specifically, this embodiment implements an incremental learning mechanism, continuously updating the memory library content based on the evaluation of processing results: expert combinations with high-quality results (i.e., expert combinations with scores greater than a preset scoring threshold) are reinforced in memory, while low-quality results may be weakened or removed. Due to the limited capacity of the memory library, this embodiment adopts an elimination strategy based on time decay and frequency of use to retain the most valuable historical experience. The system also supports memory distillation, extracting the regularities of expert activation patterns from the memory library and incorporating them into router parameters to achieve implicit knowledge transfer and improve the system's initial performance when processing new tasks. To address the dynamically changing task distribution, this embodiment designs an adaptive memory update mechanism that adjusts the memory strategy based on changes in the task distribution, ensuring that the system continuously adapts to environmental changes.
[0081] It can be seen that this application realizes the division of labor and cooperation of expert models by setting up a layered expert model structure and utilizing the task processing strategy of serial processing between different layers and parallel processing within the same layer, thereby avoiding the problem of a sharp increase in computing resources caused by adopting a single-level expert model architecture; by creating expert models according to the complexity of the tasks, the adaptability of the expert models to the tasks is improved, and the problem of waste of computing resources is avoided; by determining the expert activation sequence corresponding to the image editing task and using the adapted expert model for task processing, the consistency between the expert model for task processing and the image editing task is guaranteed, so that this solution can adapt to and handle complex and changing task requirements.
[0082] Based on the above embodiments, this application describes the overall process of processing image editing tasks using different expert models. In order to make the technical solution in this application more complete, the application will now explain the process of distributed processing of image editing tasks by expert models at different levels. Figure 3 As shown, the embodiment of the present invention discloses a process for collaboratively processing an image editing task using different expert models, including:
[0083] Step S21, obtain the target image editing task, and create a target number of initial expert models at each level of the target task processing system according to the task complexity of the target image editing task; wherein the target task processing system includes a macro layer, an intermediate layer and a micro layer.
[0084] In this embodiment, it is necessary to create a target number of initial expert models at the macro layer, intermediate layer and micro layer respectively according to the load of the task. In a specific embodiment, the FLUX.1 basic model is first initialized, and the task is evaluated by the task complexity evaluation module. Four expert adapters are created for the macro layer, namely the initial expert model (task understanding, style judgment, content planning, and resource allocation), eight expert adapters are created for the intermediate layer (text feature extraction, image feature transformation, layout processing, style conversion, object generation, background synthesis, color adjustment, and composition optimization), and six expert adapters are created for the micro layer (detail enhancement, edge processing, texture optimization, consistency check, quality assessment, and artistic enhancement). The rank of each LoRA adapter is set to 32, which accounts for approximately 0.1% of the parameters of the basic model.
[0085] Step S22: training each of the initial expert models in the target task processing system to obtain a corresponding trained expert model, and determining a target expert model corresponding to the target image editing task from each of the trained expert models.
[0086] Step S23: Analyze the target image editing task based on each target expert model in the macro layer to determine the target processing strategy corresponding to the target image editing task; and use the target processing strategy and each target expert model in the intermediate layer to perform feature transformation on the target image editing task to obtain corresponding intermediate results.
[0087] This embodiment adopts a multi-level expert structure, where experts at each level work together to complete complex task processing. The system mainly includes three levels of expert modules: macro layer, middle layer and micro layer, each layer is responsible for processing at a different level of abstraction.
[0088] The macro-level expert cluster (i.e., the target expert models within the macro layer) is responsible for global feature processing and task understanding. These experts include experts in task type identification, global strategy planning, and resource allocation (i.e., target processing strategies). These experts receive raw input, generate task representation vectors, and determine the processing strategy and activation patterns for subsequent-level experts. Macro-level experts typically employ a lightweight architecture to efficiently handle diverse input tasks, such as content analysis, feature transformation, and decision support.
[0089] The mid-level expert group is responsible for core feature conversion and executing key operations. These experts include regional feature conversion experts, logical relationship processing experts, and modal conversion experts. These experts receive guidance from the macro-level, perform specific feature transformation operations, and generate intermediate representations (i.e., intermediate results). The mid-level experts typically outnumber the macro-level experts to accommodate diverse feature conversion needs, with each expert specializing in a specific type of conversion operation.
[0090] Step S24: Utilize each target expert model in the micro layer to adjust the intermediate result to process the target image editing task.
[0091] In this embodiment, a micro-level expert group is responsible for detail optimization and quality improvement, including edge detail processing experts, consistency check experts, and quality enhancement experts. These experts receive the output of the intermediate layer and make fine adjustments to ensure the coherence, naturalness, and high-quality characteristics of the final output. Micro-level experts typically adopt a high-precision but moderately parameterized structure to balance detail processing capabilities and computational efficiency. This three-layer expert model achieves parallel processing within layers and serial processing between layers, improving the processing efficiency of image editing tasks while avoiding the problem of excessive expansion of computing resources.
[0092] Furthermore, the image editing task processing method of this embodiment further includes determining task feature information for a target image editing task, fine-tuning each target expert model corresponding to the target image editing task based on the task feature information to obtain a corresponding fine-tuned expert model, and controlling each fine-tuned expert model to collaboratively process the target image editing task. Specifically, fine-tuning is performed specifically for a specific creative style (e.g., the Ghibli animation style) to optimize the collaborative work efficiency among relevant experts, enabling them to complement and enhance each other when working in parallel within the same layer, thereby increasing the success rate of creative work in a specific style.
[0093] In addition, in this embodiment, special training is conducted for complex tasks that require precise control, strengthening information transmission between levels and ensuring that the strategic decisions of experts activated at the macro level can be accurately transmitted to the corresponding experts at the meso and micro levels, forming a consistent processing link.
[0094] For the specific process of the above step S22, reference may be made to the corresponding contents disclosed in the above embodiment, which will not be described again here.
[0095] It can be seen that this application realizes the division of labor and cooperation of expert models by setting up a layered expert model structure and utilizing the task processing strategy of serial processing between different layers and parallel processing within the same layer, thereby avoiding the problem of a sharp increase in computing resources caused by adopting a single-level expert model architecture; by creating expert models according to the complexity of the tasks, the adaptability of the expert models to the tasks is improved, and the problem of waste of computing resources is avoided; by determining the expert activation sequence corresponding to the image editing task and using the adapted expert model for task processing, the consistency between the expert model for task processing and the image editing task is guaranteed, so that this solution can adapt to and handle complex and changing task requirements.
[0096] See also Figure 4 As shown, the embodiment of the present invention discloses an image editing task processing device based on a hybrid expert model, comprising:
[0097] The complexity analysis module 11 is used to obtain a target image editing task and perform complexity analysis on the target image editing task to obtain the task complexity corresponding to the target image editing task;
[0098] an expert model creation module 12 for creating a target number of initial expert models at each level of a target task processing system according to the complexity of the task; wherein the target task processing system includes a macro level, an intermediate level, and a micro level;
[0099] an activation sequence determining module 13, configured to determine a target expert activation sequence corresponding to the target image editing task based on a similarity between the target image editing task and historical image editing tasks in a preset memory library; wherein the target expert activation sequence includes an expert combination and an expert activation order corresponding to the target image editing task;
[0100] The task processing module 14 is used to determine the target expert model from the trained expert models corresponding to each of the initial expert models according to the target expert activation sequence, activate the target expert model based on the expert activation sequence, and use each of the target expert models in different layers to process the target image editing task; wherein, the target expert models in the same layer adopt a parallel processing strategy, and the target expert models between different layers adopt a serial processing strategy.
[0101] In some specific embodiments, the expert model creation module 12 further includes:
[0102] an expert model training unit, configured to train each of the initial expert models using a first target number of training samples to obtain the trained expert models corresponding to each of the initial expert models;
[0103] A router training unit is used to train the initial router in the target task processing system using a second target number of training samples to obtain a corresponding target router; wherein the target router is used to determine a target expert activation sequence corresponding to the target image editing task.
[0104] In some specific embodiments, the activation sequence determination module 13 further includes:
[0105] a data acquisition unit, configured to acquire the historical image editing tasks, and acquire the historical feature vectors corresponding to each of the historical image editing tasks, the historical expert activation sequences, and the historical task processing scores corresponding to each of the historical expert activation sequences;
[0106] A memory library construction unit is used to construct the preset memory library using the historical feature vector, the historical expert activation sequence and the historical task processing score.
[0107] In some specific embodiments, the task processing module 14 may specifically include:
[0108] a processing strategy determining unit, configured to analyze the target image editing task based on each target expert model in the macro layer to determine a target processing strategy corresponding to the target image editing task;
[0109] A task processing unit is used to use the target processing strategy and each target expert model in the intermediate layer to perform feature transformation on the target image editing task to obtain a corresponding intermediate result, and use each target expert model in the micro layer to adjust the intermediate result to process the target image editing task.
[0110] In some specific embodiments, the activation sequence determination module 13 may specifically include:
[0111] a similarity determination unit, configured to determine whether a similarity between a target feature vector of the target image editing task and a historical feature vector of each of the historical image editing tasks in the preset memory library is less than a preset similarity threshold;
[0112] an activation sequence determining unit, configured to, if the similarity between the target feature vector and any historical feature vector is not less than a preset similarity threshold, determine whether a historical task processing score corresponding to any historical feature vector is lower than a preset score threshold; and if the historical task processing score corresponding to any historical feature vector is not lower than the preset score threshold, determine the expert activation sequence corresponding to any historical feature vector as the target expert activation sequence;
[0113] The activation sequence generating unit is configured to generate the target expert activation sequence corresponding to the target image editing task by using the target route if the similarity between the target feature vector and any historical feature vector is less than the preset similarity threshold.
[0114] In some specific embodiments, the task processing module 14 further includes:
[0115] a score obtaining unit, configured to score the target expert activation sequence based on the processing results of the target image editing task by each target expert model, so as to obtain a target task processing score corresponding to the target expert sequence;
[0116] A memory update unit is used to determine whether the target task processing score is lower than the preset score threshold. If the target task processing score is not lower than the preset score threshold, the target feature vector, the target expert activation sequence and the target task processing score corresponding to the target image editing task are stored in the preset memory to update the preset memory.
[0117] In some specific embodiments, the image editing task processing apparatus further includes:
[0118] The expert model fine-tuning module is used to determine the task feature information of the target image editing task, and fine-tune each target expert model corresponding to the target image editing task according to the task feature information to obtain the corresponding fine-tuned expert model, and control each fine-tuned expert model to collaboratively process the target image editing task.
[0119] Furthermore, the embodiment of the present application also discloses an electronic device, Figure 5 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram should not be considered as any limitation to the scope of application of the present application.
[0120] Figure 5 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps of the image editing task processing method based on the hybrid expert model disclosed in any of the aforementioned embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.
[0121] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device. The communication protocol it follows is any communication protocol that can be applied to the technical solution of this application and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world. Its specific interface type can be selected according to specific application needs and is not specifically limited here.
[0122] In addition, the memory 22 as a carrier for resource storage can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon may include an operating system 221, a computer program 222, etc., and the storage method can be temporary storage or permanent storage.
[0123] The operating system 221 is used to manage and control the hardware devices on the electronic device 20 and the computer program 222, and can be Windows Server, NetWare, Unix, Linux, etc. In addition to including a computer program capable of implementing the hybrid expert model-based image editing task processing method disclosed in any of the aforementioned embodiments and executed by the electronic device 20, the computer program 222 may further include a computer program capable of completing other specific tasks.
[0124] Furthermore, this application discloses a computer-readable storage medium for storing a computer program; wherein, when executed by a processor, the computer program implements the aforementioned method for processing image editing tasks based on a hybrid expert model. The specific steps of this method can be found in the corresponding contents disclosed in the aforementioned embodiments and will not be repeated here.
[0125] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from the other embodiments. Reference can be made to the descriptions of the identical or similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and the relevant parts can be referred to the descriptions of the methods.
[0126] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0127] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
[0128] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.
[0129] The above is a detailed introduction to the technical solution provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the ideas of the present application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.
Claims
1. A method for processing image editing tasks based on a hybrid expert model, characterized in that: include: Obtaining a target image editing task, and performing complexity analysis on the target image editing task to obtain a task complexity corresponding to the target image editing task; Creating a target number of initial expert models at each level of a target task processing system according to the complexity of the task; wherein the target task processing system includes a macro layer, an intermediate layer, and a micro layer; Determining a target expert activation sequence corresponding to the target image editing task based on a similarity between the target image editing task and historical image editing tasks in a preset memory library; wherein the preset memory library is used to store the historical expert activation sequence corresponding to the historical image editing task, and the target expert activation sequence includes an expert combination and an expert activation order corresponding to the target image editing task; According to the target expert activation sequence, a target expert model is determined from the trained expert models corresponding to each of the initial expert models, the target expert model is activated based on the expert activation sequence, and the target image editing task is processed using each of the target expert models in different layers; wherein, the target expert models in the same layer adopt a parallel processing strategy, and the target expert models between different layers adopt a serial processing strategy.
2. The image editing task processing method based on hybrid expert model according to claim 1 is characterized in that: After creating a target number of initial expert models at each level of the target task processing system according to the task complexity, the method further includes: Training each of the initial expert models using a first target number of training samples to obtain the trained expert models corresponding to each of the initial expert models; The initial router in the target task processing system is trained using a second target number of training samples to obtain a corresponding target router; wherein the target router is used to determine a target expert activation sequence corresponding to the target image editing task.
3. The image editing task processing method based on hybrid expert model according to claim 2 is characterized in that: Before determining the target expert activation sequence corresponding to the target image editing task based on the similarity between the target image editing task and the historical image editing tasks in the preset memory library, the method further includes: Obtaining the historical image editing tasks, and obtaining the historical feature vectors corresponding to each of the historical image editing tasks, the historical expert activation sequences, and the historical task processing scores corresponding to each of the historical expert activation sequences; The preset memory library is constructed using the historical feature vectors, the historical expert activation sequences, and the historical task processing scores.
4. The image editing task processing method based on hybrid expert model according to claim 1, characterized in that: The processing of the target image editing task by using the target expert models in different layers includes: Analyzing the target image editing task based on each target expert model in the macro layer to determine a target processing strategy corresponding to the target image editing task; The target processing strategy and each target expert model in the intermediate layer are used to perform feature transformation on the target image editing task to obtain a corresponding intermediate result, and the intermediate result is adjusted using each target expert model in the micro layer to process the target image editing task.
5. The image editing task processing method based on hybrid expert model according to claim 3 is characterized in that: The step of determining a target expert activation sequence corresponding to the target image editing task based on the similarity between the target image editing task and historical image editing tasks in a preset memory library includes: Determining whether the similarity between the target feature vector of the target image editing task and the historical feature vectors of each of the historical image editing tasks in the preset memory library is less than a preset similarity threshold; If the similarity between the target feature vector and any historical feature vector is not less than the preset similarity threshold, then determining whether the historical task processing score corresponding to any historical feature vector is lower than the preset score threshold; if the historical task processing score corresponding to any historical feature vector is not lower than the preset score threshold, then determining the expert activation sequence corresponding to any historical feature vector as the target expert activation sequence; If the similarity between the target feature vector and any historical feature vector is less than the preset similarity threshold, the target expert activation sequence corresponding to the target image editing task is generated by using the target routing.
6. The image editing task processing method based on hybrid expert model according to claim 5 is characterized in that: After processing the target image editing task using the target expert models in different layers, the method further includes: Scoring the target expert activation sequence based on the processing results of the target image editing task by each target expert model to obtain a target task processing score corresponding to the target expert sequence; Determine whether the target task processing score is lower than the preset score threshold. If the target task processing score is not lower than the preset score threshold, store the target feature vector, the target expert activation sequence and the target task processing score corresponding to the target image editing task into the preset memory library to update the preset memory library.
7. The image editing task processing method based on a hybrid expert model according to any one of claims 1 to 6, characterized in that: Also includes: Determine the task feature information of the target image editing task, and fine-tune each of the target expert models corresponding to the target image editing task according to the task feature information to obtain a corresponding fine-tuned expert model, and control each of the fine-tuned expert models to collaboratively process the target image editing task.
8. An image editing task processing device based on a hybrid expert model, characterized in that: include: A complexity analysis module is used to obtain a target image editing task and perform complexity analysis on the target image editing task to obtain a task complexity corresponding to the target image editing task; an expert model creation module, configured to create a target number of initial expert models at each level of a target task processing system according to the complexity of the task; wherein the target task processing system includes a macro level, an intermediate level, and a micro level; an activation sequence determination module, configured to determine a target expert activation sequence corresponding to the target image editing task based on a similarity between the target image editing task and historical image editing tasks in a preset memory library; wherein the target expert activation sequence includes an expert combination and an expert activation order corresponding to the target image editing task; A task processing module is used to determine the target expert model from the trained expert models corresponding to each of the initial expert models according to the target expert activation sequence, activate the target expert model based on the expert activation sequence, and use each of the target expert models in different layers to process the target image editing task; wherein, the target expert models in the same layer adopt a parallel processing strategy, and the target expert models between different layers adopt a serial processing strategy.
9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the image editing task processing method based on the hybrid expert model according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that Used to store a computer program, which, when executed by a processor, implements the image editing task processing method based on a hybrid expert model as described in any one of claims 1 to 7.