Model fusion method and device, equipment and storage medium
By weighted summing multiple model parameters to build a fusion model, the problem of high delay and power consumption in real-time processing tasks in the prior art is solved, and efficient model fusion effect is achieved.
Patent Information
- Application Number
- CN202411026349.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-29
- Publication Date
- 2025-07-25
AI Technical Summary
The existing model fusion method has large delay and power consumption problems in real-time processing tasks, especially in the end-side equipment.
By obtaining the model parameters of multiple models to be fused, determining the weight coefficients of each model and performing weight summing, building a fusion model, the fusion effect of multiple models is obtained by just one inference.
It reduces the operating power consumption of the fusion model, improves the operating efficiency of model fusion, and is suitable for real-time processing tasks of end-side equipment.
Smart Images
Figure CN120372522A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer technologies, and in particular, to a model fusion method, apparatus, device, and storage medium. Background Art
[0002] Model fusion refers to the technology of combining the prediction results of multiple independent models to obtain more accurate and stable prediction results. It has been widely applied in the fields of machine learning and data mining, and with the development of technologies, the methods of model fusion are also constantly evolving and improving.
[0003] In related technologies, the development of model fusion can be divided into different stages such as simple averaging and weighted averaging, voting-based fusion, stacking fusion, etc. However, these model fusion methods all require multiple inferences and then fuse the results after inference. When dealing with tasks that require real-time processing, this method has a relatively large latency and also requires greater computing power for inference. In end-side devices, these methods will cause greater power consumption and lower operating efficiency. Summary of the Invention
[0004] The present disclosure aims to at least partly solve one of the technical problems in the related technologies.
[0005] To this end, the first objective of the present disclosure is to propose a model fusion method to reduce the operating power consumption of the fusion model and improve the operating efficiency of the fusion model.
[0006] The second objective of the present disclosure is to propose a model fusion apparatus.
[0007] The third objective of the present disclosure is to propose an electronic device.
[0008] The fourth objective of the present disclosure is to propose a computer-readable storage medium.
[0009] The fifth objective of the present disclosure is to propose a computer program product.
[0010] To achieve the above objectives, the first aspect embodiment of the present disclosure proposes a model fusion method, including:
[0011] Obtaining the model parameters of each model to be fused in the set of models to be fused, where the tendencies among different models to be fused in the set of models to be fused are different;
[0012] Determining the weight coefficient corresponding to each model to be fused, and performing weighted summation on the model parameters according to the weight coefficient to obtain the fused model parameters;
[0013] Constructing a fusion model according to the fused model parameters.
[0014] Optionally, the model to be fused is an image denoising model, the model to be fused is an image denoising model, and the fused model is an image denoising fused model. After constructing the fused model according to the fused model parameters, the method further includes:
[0015] Obtain the image to be denoised;
[0016] Control the image denoising fused model to perform model inference on the image to be denoised based on the fused model parameters, and obtain the denoised image corresponding to the image to be denoised.
[0017] Optionally, before obtaining the model parameters of each model to be fused in the model to be fused set, the method further includes:
[0018] Determine the initial model to be fused and the model training strategy set;
[0019] Train the initial model to be fused respectively using each model training strategy in the model training strategy set to obtain a set of models to be fused, where the models to be fused in the set of models to be fused correspond one-to-one with the model training strategies.
[0020] Optionally, the determining the weight coefficient corresponding to each model to be fused includes:
[0021] Obtain the weight coefficient input for any model to be fused in the set of models to be fused.
[0022] Optionally, the determining the weight coefficient corresponding to each model to be fused includes:
[0023] Determine the initial weight coefficient corresponding to each model to be fused;
[0024] Perform weighted summation on the model parameters according to the initial weight coefficient to obtain the initial fused model parameters;
[0025] Construct an initial fused model according to the initial fused model parameters, and use the satisfaction of the initial fused model with the model fusion requirements as the optimization goal to optimize and solve the initial weight coefficient to obtain the weight coefficient corresponding to each model to be fused.
[0026] Optionally, the optimizing and solving the initial weight coefficient includes:
[0027] Use the gradient descent method to optimize and solve the initial weight coefficient.
[0028] Optionally, the sum of the weight coefficients corresponding to all models to be fused in the set of models to be fused is 1.
[0029] To achieve the above object, an embodiment of the second aspect of the present disclosure provides a model fusion device, including:
[0030] A parameter acquisition unit, configured to acquire model parameters of each model to be fused in a set of models to be fused, where different models to be fused in the set of models to be fused have different tendencies;
[0031] A parameter fusion unit, configured to determine a weight coefficient corresponding to each model to be fused, and perform weighted summation on the model parameters according to the weight coefficient to obtain fused model parameters;
[0032] A model construction unit, configured to construct a fusion model according to the fused model parameters.
[0033] To achieve the above object, an embodiment of the third aspect of the present disclosure provides an electronic device, including: a processor, and a memory communicatively connected to the processor;
[0034] The memory stores computer-executable instructions;
[0035] The processor executes the computer-executable instructions stored in the memory to implement the method shown in any one of the foregoing first aspects.
[0036] To achieve the above object, an embodiment of the fourth aspect of the present disclosure provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the computer-executable instructions are executed by a processor, they are used to implement the method shown in any one of the foregoing first aspects.
[0037] To achieve the above object, an embodiment of the fifth aspect of the present disclosure provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the method shown in any one of the foregoing first aspects.
[0038] In summary, the method, device, equipment, and storage medium provided by the present disclosure weight multiple model parameters with different tendencies to obtain fused model parameters, and construct a new fusion model according to the fused model parameters. Therefore, the constructed fusion model can obtain the effect after fusing multiple models with different tendencies only through one inference, and there is no need to perform inference on each model with different tendencies and then weight to obtain the final output, which can reduce the running power consumption of the fusion model and improve the running efficiency of the fusion model.
[0039] Additional aspects and advantages of the present disclosure will be given in part in the following description, become apparent in part from the following description, or be learned through the practice of the present disclosure. Description of the Drawings
[0040] The above and / or additional aspects and advantages of the present disclosure will become apparent and be readily understood from the following description of embodiments in conjunction with the accompanying drawings, where:
[0041] Figure 1 It is a schematic flowchart of a model fusion method provided by an embodiment of the present disclosure;
[0042] Figure 2 It is a schematic flowchart of a model fusion method provided by another embodiment of the present disclosure;
[0043] Figure 3 It is a comparison schematic diagram of image denoising provided by an embodiment of the present disclosure;
[0044] Figure 4 It is a schematic structural diagram of a model fusion device provided by an embodiment of the present disclosure. Detailed Embodiments
[0045] The embodiments of the present disclosure will be described in detail below. Examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements with the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to explain the present disclosure and should not be construed as limiting the present disclosure.
[0046] With the development of science and technology, model fusion has been widely applied in various industries. For example, in the task of image denoising, it is often necessary to denoise an image to a result that conforms to the human eye's perception. Therefore, it is necessary to balance the relationship between noise and details in the model fusion process. However, due to various factors, this balance is not easy to control. The effects presented by different artificial intelligence (AI) denoising models trained using different training strategies or datasets may have strong denoising but weak details, or many details but heavy noise, all of which do not conform to the subjective perception of the human eye. In this case, by fusing different parameter AI denoising models of multiple identical networks, an AI denoising model that meets the delivery requirements can be obtained. In addition, model fusion can also be applied to other similar underlying vision tasks, such as image dehazing, de-raining, and image super-resolution.
[0047] In the related art, there are many methods for model fusion in similar underlying vision tasks. Among them, the simplest method is to assign different weights to the outputs of multiple models with different parameters and then add them together. This method is often used in super-resolution tasks. In addition, there is also the Self-ensemble method. Specifically, it involves applying rotations with different angles and horizontal flips to low-quality images, then inputting these images into the network model, performing corresponding inverse transformations on the high-quality images restored by the network to obtain the outputs, and finally determining the final prediction result through the average or median of these outputs. Additionally, there are some more complex methods, such as the fusion method based on k-means clustering, which performs k-means clustering on the outputs of the model to achieve a better fusion effect.
[0048] However, the above-mentioned model fusion methods all require multiple inferences and then fuse the results after inference. When dealing with tasks that require real-time processing, this method has a relatively large time delay and also requires more computing power for inference. In end-side devices, these methods will cause greater power consumption and lower operating efficiency.
[0049] The following will explain the present disclosure in detail with specific embodiments.
[0050] In the first embodiment, as Figure 1 shown, Figure 1 is a schematic flowchart of a model fusion method provided by an embodiment of the present disclosure. This method can be implemented depending on a computer program and can run on a device for model fusion. This computer program can be integrated in an application or run as an independent tool application.
[0051] Among them, the model fusion device can be an electronic device with model fusion capabilities.
[0052] Among them, this model fusion method can be executed by an electronic device.
[0053] Exemplarily, this model fusion method includes the following steps:
[0054] S101, obtain the model parameters of each model to be fused in the set of models to be fused;
[0055] According to some embodiments, the model to be fused refers to the model that needs to perform model fusion. The set of models to be fused includes multiple models to be fused.
[0056] In some embodiments, the model parameters refer to the configuration variables inside the model, and its value can be estimated using data.
[0057] According to some embodiments, the tendencies among different models to be fused in the set of models to be fused are different.
[0058] In some embodiments, the tendency of a model refers to the systematic preference or discrimination of the model for a specific category, feature, or group when making predictions or decisions due to various factors such as training data, algorithm design, model architecture, or external environment.
[0059] S102. Determine the weight coefficients corresponding to each model to be fused, and perform weighted summation on the model parameters according to the weight coefficients to obtain the fused model parameters.
[0060] According to some embodiments, the weight coefficients corresponding to different models to be fused can be the same or different.
[0061] In some embodiments, the weight coefficient corresponding to the same model to be fused does not specifically refer to a certain fixed coefficient. This weight coefficient can be adjusted according to the actual application scenario, for example.
[0062] According to some embodiments, the fused model parameters refer to the parameters obtained by performing weighted summation on the model parameters according to the weight coefficients.
[0063] S103. Construct a fused model according to the fused model parameters.
[0064] According to some embodiments, a fused model refers to a model that uses the fused model parameters as its model parameters.
[0065] In summary, the method provided in this embodiment weights the model parameters of multiple models with different tendencies to obtain the fused model parameters, and constructs a new fused model according to the fused model parameters. Therefore, the constructed fused model can obtain the fused effect of multiple models with different tendencies with only one inference, without the need to perform inferences on each model with different tendencies and then weight them to obtain the final output, which can reduce the operating power consumption of the fused model and improve the operating efficiency of the fused model.
[0066] This embodiment also provides another model fusion method. This method can be executed by an electronic device.
[0067] Exemplarily, this model fusion method can include the following steps:
[0068] S201. Determine the initial models to be fused and the set of model training strategies.
[0069] According to some embodiments, the initial models to be fused and the set of model training strategies can be selected according to the actual application scenario.
[0070] For example, in the field of image denoising, the initial model to be fused can be the initial model under the image denoising network structure, and the model training strategies in the model training strategy set can be the training strategies applied to the initial image denoising model.
[0071] In some embodiments, the model training strategy can include, for example, the model training steps and the sample set used when training the model.
[0072] S202. Train the initial model to be fused respectively using each model training strategy in the model training strategy set to obtain a set of models to be fused;
[0073] According to some embodiments, the models to be fused in the set of models to be fused correspond one-to-one with the model training strategies.
[0074] Taking one scenario as an example, Figure 2 is a schematic flowchart of a model fusion method provided by an embodiment of the present disclosure. As Figure 2 shown, the initial model to be fused can be trained using model training strategy A to obtain the model to be fused 1, the initial model to be fused can be trained using model training strategy B to obtain the model to be fused 2, and the initial model to be fused can be trained using model training strategy C to obtain the model to be fused 3.
[0075] S203. Obtain the model parameters of each model to be fused in the set of models to be fused;
[0076] Taking one scenario as an example, as Figure 2 shown, all the parameters in the model to be fused 1 can be obtained to get the model parameter A of the model to be fused 1; all the parameters in the model to be fused 2 can be obtained to get the model parameter B of the model to be fused 2; all the parameters in the model to be fused 3 can be obtained to get the model parameter C of the model to be fused 3.
[0077] In some embodiments, since each model to be fused is trained using a different model training strategy, therefore, the model parameters of different models to be fused are not the same, and the effects corresponding to each model parameter have different tendencies.
[0078] For example, when the model to be fused is an image denoising model, the tendency of the model to be fused 1 can be more noise and more details, the tendency of the model to be fused 2 can be less noise and less details, and the tendency of the model to be fused 3 can be more noise and less details.
[0079] S204. Determine the weight coefficient corresponding to each model to be fused;
[0080] According to some embodiments, in the process of determining the weight coefficient corresponding to each model to be fused, the weight coefficient input for any model to be fused in the set of models to be fused can be obtained. In this case, the user can manually adjust the weight coefficient corresponding to each model to be fused, which is more suitable for the weighting of a small number of model parameters, and the flexibility of model fusion is higher.
[0081] According to some embodiments, in the process of determining the weight coefficient corresponding to each model to be fused, the initial weight coefficient corresponding to each model to be fused can also be determined; the initial model parameters after fusion are obtained by weighted summation of the model parameters according to the initial weight coefficient; an initial fusion model is constructed according to the initial model parameters after fusion, and the initial weight coefficient is optimized and solved with the requirement that the initial fusion model meets the model fusion requirement, so as to obtain the weight coefficient corresponding to each model to be fused. In this case, there is no need for the user to manually adjust the weight coefficient, which is more suitable for the weighting of a large number of model parameters, and there is no need to consume a large amount of time to retrain and fine-tune the model.
[0082] In some embodiments, the model fusion requirement can be adjusted according to the actual application scenario. For example, in the field of image denoising, the model fusion requirement can be that the noise of the image obtained after image denoising using the initial fusion model is within a preset noise range, and the details are within a preset detail range.
[0083] In some embodiments, the gradient descent method can be used to optimize and solve the initial weight coefficient.
[0084] It should be noted that in the process of determining the weight coefficient corresponding to each model to be fused, the weight coefficient input for any model to be fused in the set of models to be fused can also be obtained first, and then the input weight coefficient is optimized and solved; or the initial weight coefficient can be optimized and solved first, and then the user manually adjusts the solved weight coefficient.
[0085] According to some embodiments, the sum of the weight coefficients corresponding to all models to be fused in the set of models to be fused is 1. For example, as Figure 2 shown, weight coefficient A + weight coefficient B + weight coefficient C = 1.
[0086] S205, perform weighted summation on the model parameters according to the weight coefficient to obtain the model parameters after fusion;
[0087] Taking one scenario as an example, as Figure 2 shown, the model parameters after fusion can be shown as the following formula:
[0088] par_A * weight_A + par_B * weight_B + par_C * weight_C
[0089] Among them, par_A represents model parameter A; weight_A represents weight coefficient A; par_B represents model parameter B; weight_B represents weight coefficient B; par_C represents model parameter C; weight_C represents weight coefficient C.
[0090] S206. Construct a fusion model according to the fused model parameters.
[0091] According to some embodiments, such as Figure 2 As shown, the fused output after the fusion model performs model inference can be, for example, as shown in the following formula:
[0092] fused_output = model(x, par_A * weight_A + par_B * weight_B + par_C * weight_C)
[0093] Among them, fused_output is the fused output; x represents the input of the fusion model.
[0094] Taking a scenario as an example, when the model to be fused is an image denoising model, the obtained fusion model is an image denoising fusion model, and this image denoising fusion model can be applied to image denoising. The input of the model can be a low-quality image or a degraded image with noise.
[0095] In some embodiments, the number of models to be fused is at least three; when the number of models to be fused is less than 3, manual fusion can be directly performed.
[0096] In some embodiments, when using the image denoising fusion model for image denoising, a to-be-denoised image can be obtained; control the image denoising fusion model to perform model inference on the to-be-denoised image based on the fused model parameters to obtain the denoised image corresponding to the to-be-denoised image.
[0097] For example, Figure 3 is a comparison schematic diagram of an image denoising provided by an embodiment of the present disclosure. As Figure 3 shown, the left image is the image obtained after the image denoising model A with a tendency of more noise and more details processes the to-be-denoised image, the right image is the image obtained after the image denoising model B with a tendency of less noise and less details processes the to-be-denoised image, and the middle image is the image obtained after the image denoising fusion model obtained by fusing the image denoising model A and the image denoising model B using the model fusion method proposed in the embodiment of the present disclosure processes the to-be-denoised image.
[0098] As Figure 3As shown, the image denoising fusion model obtained by performing model fusion using the model fusion method proposed in the embodiments of the present disclosure can obtain model parameters with relatively balanced noise details, can obtain an image denoising fusion model that conforms to the human eye perception, and does not need to consume a large amount of time to train and fine-tune the AI model again. It can be applied to the underlying vision tasks processed by small models on the edge side, and can more efficiently adjust the model output to obtain an effect that conforms to the human eye perception.
[0099] In summary, for the method provided in this embodiment, first, by determining the initial model to be fused and the model training strategy set; respectively training the initial model to be fused using each model training strategy in the model training strategy set to obtain a set of models to be fused; thus, different models to be fused in the set of models to be fused can have different tendencies. Then, by obtaining the model parameters of each model to be fused in the set of models to be fused; determining the weight coefficient corresponding to each model to be fused; performing weighted summation on the model parameters according to the weight coefficient to obtain the fused model parameters; and constructing a fusion model according to the fused model parameters. Therefore, the constructed fusion model can obtain the effect after the fusion of multiple models with different tendencies with only one inference, and it is not necessary to perform inference on each model with different tendencies and then perform weighted summation to obtain the final output, which can reduce the operating power consumption of the fusion model and improve the operating efficiency of the fusion model.
[0100] To implement the above embodiments, the present disclosure also proposes a model fusion device.
[0101] As Figure 4 shown, the model fusion device 400 includes:
[0102] A parameter acquisition unit 401, configured to acquire the model parameters of each model to be fused in the set of models to be fused, where different models to be fused in the set of models to be fused have different tendencies;
[0103] A parameter fusion unit 402, configured to determine the weight coefficient corresponding to each model to be fused, and perform weighted summation on the model parameters according to the weight coefficient to obtain the fused model parameters;
[0104] A model construction unit 403, configured to construct a fusion model according to the fused model parameters.
[0105] Optionally, the model to be fused is an image denoising model, the model to be fused is an image denoising model, the fusion model is an image denoising fusion model. After constructing the fusion model according to the fused model parameters, the model construction unit 403 is further configured to:
[0106] Acquire the image to be denoised;
[0107] Based on the fused model parameters, the control image denoising fusion model performs model inference on the image to be denoised, and obtains the denoised image corresponding to the image to be denoised.
[0108] Optionally, before obtaining the model parameters of each to-be-fused model in the to-be-fused model set, the parameter acquisition unit 401 is further configured to:
[0109] Determine the initial to-be-fused model and the model training strategy set;
[0110] Train the initial to-be-fused model respectively using each model training strategy in the model training strategy set to obtain the to-be-fused model set, where the to-be-fused models in the to-be-fused model set correspond to the model training strategies one by one.
[0111] Optionally, when the parameter fusion unit 402 is used to determine the weight coefficient corresponding to each to-be-fused model, it is specifically configured to:
[0112] Obtain the weight coefficient input for any to-be-fused model in the to-be-fused model set.
[0113] Optionally, when the parameter fusion unit 402 is used to determine the weight coefficient corresponding to each to-be-fused model, it is specifically configured to:
[0114] Determine the initial weight coefficient corresponding to each to-be-fused model;
[0115] Perform weighted summation on the model parameters according to the initial weight coefficients to obtain the initial fused model parameters;
[0116] Construct an initial fusion model according to the initial fused model parameters, and take the initial fusion model meeting the model fusion requirements as the optimization objective, and optimize and solve the initial weight coefficients to obtain the weight coefficient corresponding to each to-be-fused model.
[0117] Optionally, when the parameter fusion unit 402 is used to optimize and solve the initial weight coefficients, it is specifically configured to:
[0118] Use the gradient descent method to optimize and solve the initial weight coefficients.
[0119] Optionally, the sum of the weight coefficients corresponding to all the to-be-fused models in the to-be-fused model set is 1.
[0120] It should be noted that the foregoing explanations of the embodiments of the model fusion method are also applicable to the model fusion device of this embodiment, and will not be elaborated here.
[0121] In summary, the device provided by the embodiments of the present disclosure obtains the fused model parameters by weighting multiple model parameters with different tendencies, and constructs a new fused model according to the fused model parameters. Therefore, the constructed fused model can obtain the effects after the fusion of multiple models with different tendencies through only one inference, without the need to perform inferences on each model with different tendencies and then weight them to obtain the final output, which can reduce the operating power consumption of the fused model and improve the operating efficiency of the fused model.
[0122] To implement the above embodiments, the present disclosure also provides an electronic device, including: a processor, and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the method provided by the foregoing embodiments.
[0123] To implement the above embodiments, the present disclosure also provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the method provided by the foregoing embodiments when executed by a processor.
[0124] To implement the above embodiments, the present disclosure also provides a computer program product including a computer program, which implements the method provided by the foregoing embodiments when executed by a processor.
[0125] The collection, storage, use, processing, transmission, provision, and disclosure of the user's personal information involved in the present disclosure all comply with the provisions of relevant laws and regulations and do not violate public order and good customs.
[0126] It should be noted that personal information from users should be collected for legal and reasonable purposes and not shared or sold outside of these legal uses. In addition, such collection / sharing should be carried out after obtaining the informed consent of the user, including but not limited to notifying the user to read the user agreement / user notice and sign an agreement / authorization including authorizing relevant user information before the user uses the function. In addition, any necessary steps should be taken to protect and safeguard access to such personal information data and ensure that others with access to personal information data comply with their privacy policies and procedures.
[0127] The present disclosure anticipates providing embodiments that allow users to selectively block the use or access of personal information data. That is, the present disclosure anticipates providing hardware and / or software to prevent or block access to such personal information data. Once personal information data is no longer needed, the risk can be minimized by restricting data collection and deleting the data. In addition, when applicable, personal identifiers are removed from such personal information to protect the privacy of the user.
[0128] In the description of the foregoing embodiments, the descriptions referring to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, without contradiction, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.
[0129] In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be construed as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present disclosure, "a plurality of" means at least two, such as two, three, etc., unless otherwise specifically defined.
[0130] Any process or method description in a flowchart or described in other ways herein may be understood to represent a module, segment, or portion of code including one or more executable instructions for implementing a customized logic function or process, and the scope of the preferred embodiments of the present disclosure includes additional implementations, where the functions may be executed in a substantially simultaneous manner or in a reverse order according to the involved functions, rather than in the order shown or discussed, which should be understood by those skilled in the art to which the embodiments of the present disclosure belong.
[0131] The logic and / or steps represented in the flowchart or otherwise described herein can be considered as a definable sequence of executable instructions for implementing logical functions, which can be embodied specifically in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection portion with one or more wirings (electronic device), a portable computer diskette (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can even be paper or other suitable media on which a program can be printed, as the program can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpretation, or other appropriate processing as necessary, and then stored in a computer memory.
[0132] It should be understood that various parts of the present disclosure can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or a combination of the following techniques known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.
[0133] Those of ordinary skill in the art of this technology can understand that all or part of the steps carried by the methods of the above embodiments can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiments.
[0134] In addition, in each embodiment of the present disclosure, each functional unit may be integrated into a processing module, or each unit may exist physically alone, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. When the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.
[0135] The above-mentioned storage medium may be a read-only memory, a magnetic disk, an optical disc, etc. Although the embodiments of the present disclosure have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present disclosure.
Claims
1. A model fusion method, characterized in that Including: Obtain the model parameters of each model to be fused in the set of models to be fused, where the tendencies among different models to be fused in the set of models to be fused are different; Determine the weight coefficient corresponding to each model to be fused, and perform weighted summation on the model parameters according to the weight coefficient to obtain the fused model parameters; Construct a fusion model based on the fused model parameters.
2. The method according to claim 1, characterized in that, The model to be fused is an image denoising model, and the fusion model is an image denoising fusion model. After constructing the fusion model based on the fused model parameters, the method further includes: Obtain the image to be denoised; Control the image denoising fusion model to perform model inference on the image to be denoised based on the fused model parameters to obtain the denoised image corresponding to the image to be denoised.
3. The method according to claim 1, wherein Before obtaining the model parameters of each model to be fused in the set of models to be fused, the method further includes: Determine the initial model to be fused and the set of model training strategies; Train the initial model to be fused respectively using each model training strategy in the set of model training strategies to obtain the set of models to be fused, where the models to be fused in the set of models to be fused correspond one-to-one with the model training strategies.
4. The method according to claim 1, wherein The determining the weight coefficient corresponding to each model to be fused includes: Obtain the weight coefficient input for any model to be fused in the set of models to be fused.
5. The method according to claim 1, characterized in that, The determining the weight coefficient corresponding to each model to be fused includes: Determine the initial weight coefficient corresponding to each model to be fused; Perform weighted summation on the model parameters according to the initial weight coefficient to obtain the fused initial model parameters; Construct an initial fusion model based on the fused initial model parameters, and take the satisfaction of the initial fusion model with the model fusion requirements as the optimization objective, and perform optimization solution on the initial weight coefficient to obtain the weight coefficient corresponding to each model to be fused.
6. The method according to claim 5, wherein The performing optimization solution on the initial weight coefficient includes: Perform optimization solution on the initial weight coefficient using the gradient descent method.
7. The method according to claim 1, wherein The sum of the weight coefficients corresponding to all models to be fused in the set of models to be fused is 1.
8. A model fusion device, characterized in that, Including: A parameter acquisition unit for obtaining the model parameters of each model to be fused in the set of models to be fused, where the tendencies among different models to be fused in the set of models to be fused are different; A parameter fusion unit for determining the weight coefficient corresponding to each model to be fused, and performing weighted summation on the model parameters according to the weight coefficient to obtain the fused model parameters; A model construction unit for constructing a fusion model based on the fused model parameters.
9. An electronic device, characterized in that, Including: A processor and a memory communicatively connected to the processor; The memory stores computer execution instructions; The processor executes the computer execution instructions stored in the memory to implement the method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, Computer execution instructions are stored in the computer-readable storage medium, and when the computer execution instructions are executed by the processor, they are used to implement the method according to any one of claims 1 to 7.