Selection method and device of generative model, equipment and medium

Abstract

The embodiment of the invention relates to a selection method and device for a generative model, equipment and a medium. The method comprises the steps that target prompt information is acquired; wherein the target prompt information comprises target description information of to-be-generated multimedia information and model information of a plurality of candidate generation models; the candidate generation model is a model with multimedia information generation capability, and the model information comprises an evaluation result of the candidate generation model under at least one preset evaluation dimension; based on the target prompt information, selecting a target generation model from a plurality of candidate generation models through a preset language model; wherein the target generation model is used for generating target multimedia information corresponding to the target description information. According to the embodiment of the invention, the generation model which is more suitable for generating the multimedia information corresponding to the target description information can be flexibly and reliably selected, so that the generation effect of the multimedia information can be effectively guaranteed, and the user experience is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a method, device, equipment and medium for selecting a generation model. Background Art

[0002] With the popularity of generative models, generative models are used in many fields such as gaming and academia to conveniently and quickly generate the required multimedia information. There are many existing generative models, some of which are vertical models that are only used to generate specific categories of objects, and some are general models that can generate multiple categories of objects. In related technologies, the system usually directly uses the specified generative model to generate multimedia information, which has poor results and poor user experience. Summary of the invention

[0003] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a method, device, equipment and medium for selecting a generation model.

[0004] In a first aspect, an embodiment of the present disclosure provides a method for selecting a generation model, the method comprising: obtaining target prompt information; wherein the target prompt information comprises target description information of the multimedia information to be generated and model information of multiple candidate generation models; the candidate generation model is a model having the ability to generate the multimedia information, and the model information comprises an evaluation result of the candidate generation model under at least one preset evaluation dimension; based on the target prompt information, selecting a target generation model from the multiple candidate generation models through a preset language model; wherein the target generation model is used to generate the target multimedia information corresponding to the target description information.

[0005] In a second aspect, an embodiment of the present disclosure also provides a device for selecting a generation model, comprising: an information acquisition module, used to acquire target prompt information; wherein the target prompt information includes target description information of the multimedia information to be generated and model information of multiple candidate generation models; the candidate generation model is a model with the ability to generate the multimedia information, and the model information includes an evaluation result of the candidate generation model corresponding to at least one preset evaluation dimension; a model selection module, used to select a target generation model from the multiple candidate generation models through a preset language model based on the target prompt information; wherein the target generation model is used to generate the target multimedia information corresponding to the target description information.

[0006] In a third aspect, an embodiment of the present disclosure further provides an electronic device, comprising: a processor; a memory for storing executable instructions of the processor; the processor is used to read the executable instructions from the memory and execute the instructions to implement a method for selecting a generation model as provided in an embodiment of the present disclosure.

[0007] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program is used to execute the method for selecting a generation model as provided in an embodiment of the present disclosure.

[0008] In a fifth aspect, an embodiment of the present disclosure further provides a computer program product, including a computer program, which, when executed by a processor, implements the method for constructing an evaluation model or the method for evaluating model output information provided in an embodiment of the present disclosure.

[0009] The above-mentioned technical solution provided by the embodiment of the present disclosure can make use of the strong information processing capability of the language model, and based on the target prompt information (specifically including the description information of the multimedia information to be generated and the evaluation results of multiple candidate generation models under at least one preset evaluation dimension), select a target generation model from multiple candidate generation models to generate target multimedia information corresponding to the target description information. Compared with directly using a specified generation model to generate multimedia information corresponding to the target description information, the embodiment of the present disclosure can flexibly and reliably select a generation model that is more suitable for generating multimedia information corresponding to the target description information, thereby effectively ensuring the generation effect of multimedia information and improving user experience.

[0010] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0012] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0013] Figure 1 A schematic diagram of a flow chart of a method for selecting a generation model provided in an embodiment of the present disclosure;

[0014] Figure 2 A schematic diagram of the structure of an evaluation model provided in an embodiment of the present disclosure;

[0015] Figure 3 A schematic diagram of a selection process of a generation model provided in an embodiment of the present disclosure;

[0016] Figure 4 A flowchart of a method for generating multimedia information provided by an embodiment of the present disclosure;

[0017] Figure 5 A schematic diagram of the structure of a selection device for generating a model provided by an embodiment of the present disclosure;

[0018] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0019] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0020] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0021] Figure 1 The present invention provides a flow chart of a method for selecting a generation model provided by an embodiment of the present invention. The method can be performed by a device for selecting a generation model, wherein the device can be implemented by software and / or hardware and can generally be integrated in an electronic device. Figure 1 As shown, the method mainly includes the following steps S102 to S104:

[0022] Step S102, obtaining target prompt information; wherein the target prompt information includes target description information of the multimedia information to be generated and model information of multiple candidate generation models; the candidate generation model is a model with the ability to generate multimedia information, and the model information includes an evaluation result of the candidate generation model under at least one preset evaluation dimension.

[0023] The disclosed embodiments do not limit the form of multimedia information. Exemplarily, multimedia information includes text, image, audio or video. The disclosed embodiments do not limit the form and content of target description information. The target description information is mainly used to describe the multimedia information to be generated, so as to prompt the generation model to generate the required multimedia information. Generally, the more detailed and specific the target description information is, the more the multimedia information generated by the generation model can meet the requirements. The way in which the generation model generates corresponding multimedia information based on the description information can refer to the relevant technology, which will not be repeated here.

[0024] Related technologies usually directly input the target description information into a specified generation model, so that the generation model generates multimedia information corresponding to the target description information. The inventor has found through research that, for the convenience of processing, in most cases, the target description information is directly input into a general generation model to use the general model to generate multimedia information corresponding to the target description information. Although the general model can widely generate different types of objects, the generation effect is not good. In fact, there are many existing generation models, such as one or more corresponding models in various vertical fields, and different models have their own characteristics. Taking the vehicle field as an example, there are vehicle generation models that are good at generating cars, and there are also vehicle generation models that are good at generating buses; taking the animal field as an example, there are animal generation models that are good at generating cats, and there are also animal generation models that are good at generating birds... It can be understood that different generation models can generate or are good at generating multimedia information due to differences in training samples and model structures. Based on this, in order to ensure the generation effect, the embodiments of the present disclosure do not directly specify the generation model, such as directly adopting a general generation model, but select the required target generation model from multiple existing generation models (which can be open source models or self-trained models). In order to be able to reliably select a suitable target generation model, the embodiments of the present disclosure can set at least one evaluation dimension, that is, to evaluate the generation quality of multimedia information from different angles.

[0025] The evaluation dimensions corresponding to multimedia information in different forms may be different. For example, if the multimedia information is in text form, the corresponding multiple evaluation dimensions may be fluency dimension, logic dimension, content matching dimension, etc.; if the multimedia information is in image form, the corresponding multiple evaluation dimensions may be clarity dimension, aesthetic dimension, image-text matching dimension, exposure dimension, texture dimension, etc.; if the multimedia information is in audio form, the corresponding multiple evaluation dimensions may be sound quality dimension, content matching dimension, fluency dimension, etc.; if the multimedia information is in video form, the corresponding multiple evaluation dimensions may be picture dimension, content matching dimension, fluency dimension, etc. It should be noted that the above are all exemplary descriptions. In actual applications, the corresponding evaluation dimensions can be determined flexibly according to the form of multimedia information that can be generated by the generation model, and no limitation is made here. In addition, the evaluation results can be expressed in the form of evaluation grade, evaluation score, etc., which can be flexibly set.

[0026] Compared with traditional prompt information that only includes target description information of the multimedia information to be generated, the target prompt information provided by the embodiment of the present disclosure not only includes target description information, but also includes model information of multiple candidate generation models with the ability to generate multimedia information, so as to characterize the evaluation results of each candidate generation model under at least one preset evaluation dimension, which helps to flexibly and reliably select the required model in the subsequent process.

[0027] Step S104, based on the target prompt information, a target generation model is selected from a plurality of candidate generation models through a preset language model; wherein the target generation model is used to generate target multimedia information corresponding to the target description information.

[0028] Exemplarily, the language model may perform the following operations: perform task analysis on the target description information to obtain the analysis result; based on the analysis result and the model information of multiple candidate generation models, select the target generation model from multiple candidate generation models. It can be understood that the target description information is mainly used to describe the multimedia information to be generated, so as to prompt the generation model to generate the required multimedia information. Therefore, it can also be understood that the target description information is used to indicate the tasks to be performed by the generation model. Based on this, the language model performs task analysis on the target description information, and the obtained task analysis result can include the tasks that the user expects the generation model to perform, and then the target generation model matching the task can be reasonably selected from multiple candidate generation models. For example, the language model can learn the generation model that performs better under which evaluation dimension the target description information focuses on through task analysis. Specifically, the language model can determine the target evaluation dimension based on the analysis result, and determine the target generation model based on the evaluation results of multiple candidate generation models under the target evaluation dimension. Exemplarily, the target generation model can be the candidate generation model with the best evaluation result under the target evaluation dimension. For example, if the analysis result corresponding to the target description information indicates that an illustration needs to be generated, then the target evaluation dimension is the aesthetic dimension; if the analysis result corresponding to the target description information indicates that a virtual exaggerated painting (such as a horse on the shoulders of an astronaut) needs to be generated, then the target evaluation dimension is the image-text matching dimension. It can also be seen from the above description that the purpose of task analysis in the embodiment of the present disclosure is mainly to determine the task that the user expects to perform, in other words, to determine the task corresponding to the target description information, so as to ultimately select a suitable target generation model based on the task analysis result. Therefore, task analysis should not be simply understood as analyzing the execution steps of the task.

[0029] Compared with directly using a specified generation model to generate multimedia information corresponding to the target description information, the disclosed embodiment can flexibly and reliably select a more suitable generation model, thereby effectively ensuring the generation effect of the multimedia information and improving the user experience.

[0030] In some implementations, the step of obtaining target prompt information may be performed with reference to the following steps A to C:

[0031] Step A: upon receiving target description information of the multimedia information to be generated, obtaining model information of a plurality of preset generation models.

[0032] The disclosed embodiments do not limit the source of the target description information, for example, it can be determined based on multimedia demand information input by the user. When the target description information is received, it indicates that the generation model needs to be called to generate the multimedia information corresponding to the target description information. At this time, the model information of the multiple existing preset generation models can be retrieved first. That is, in actual applications, the model information of multiple preset generation models can be acquired and stored in advance. When the target description information of the multimedia information to be generated is received, the model information of the multiple stored preset generation models can be directly called.

[0033] Step B: selecting multiple candidate generation models from multiple preset generation models based on the target description information and model information of multiple preset generation models.

[0034] In practical applications, a preset generation model related to the target description information can be selected as a candidate generation model according to the model information of multiple preset generation models. In some specific implementation examples, the model information may include the evaluation results of each generation model under at least one preset evaluation dimension. In practical applications, in order to facilitate the intuitive and vivid presentation of the evaluation results, the evaluation results can be represented in the form of scores. In order to provide model information more accurately and objectively, the model information may specifically include the following three related contents: 1) description information sample 2) identifier of the preset generation model 3) evaluation results of the preset generation model corresponding to the description information sample under at least one preset evaluation dimension. Specifically, the evaluation result may indicate the quality of the multimedia information generated by the preset generation model for the description information sample under at least one preset evaluation dimension. Exemplarily, the evaluation score representation may be adopted, such as the model information is represented by a [description information sample, model identifier, score] triple pair, and the generation model is an image generation model, and the preset evaluation dimensions include clarity dimension, aesthetic dimension, and image-text matching dimension. For example, the model information can be represented in the form of [description information sample a, model identifier 1, aesthetic score 1, clarity score 1, image-text matching score 1]. On the basis of the foregoing, for example, a preset generation model that has previously generated multimedia information for description information samples similar to the target description information can be selected as a candidate generation model. This can not only effectively ensure that the candidate generation models have the ability to generate multimedia information corresponding to the target description information, but also further narrow the selection range of the generation model and improve the model selection efficiency and accuracy.

[0035] Step C, obtaining target prompt information based on the target description information and model information of multiple candidate generation models.

[0036] Based on the known target description information and the model information of multiple candidate generation models, the target prompt information can be combined in a preset format to obtain the target prompt information. The embodiment of the present disclosure does not limit the format of the target prompt information.

[0037] Through the above steps A to C, reasonable target prompt information can be obtained. The target prompt information is used to instruct the language model to select a target generation model from multiple candidate generation models based on the target description information and model information. The target generation model is used to generate multimedia information corresponding to the target description information.

[0038] In order to facilitate the understanding of the above steps A to C, the present disclosure further describes the above steps as follows:

[0039] Exemplarily, the model information of the preset generation model mentioned in the above step A can be obtained in advance through the following steps a to b:

[0040] Step a, obtaining a multimedia sample output by a preset generation model for a description information sample corresponding to the preset generation model; wherein one preset generation model corresponds to one or more description information samples, and description information samples corresponding to different preset generation models are the same or different. In practical applications, in order to facilitate comparison and evaluation, description information samples corresponding to generation models belonging to the same vertical category may be the same, and description information samples corresponding to generation models belonging to different vertical categories may be different.

[0041] Step b, based on the multimedia sample output by the preset generation model for the corresponding description information sample, obtain the evaluation result of the preset generation model under at least one preset evaluation dimension. In practical applications, the evaluation result can be obtained by manual evaluation or by using a pre-trained evaluation model.

[0042] In order to improve evaluation efficiency, in some specific implementation examples, the description information samples corresponding to the preset generation model and the multimedia samples output by the preset generation model for the corresponding description information samples can be input into the preset evaluation model, so as to output the evaluation results of the preset generation model under at least one preset evaluation dimension through the evaluation model.

[0043] For ease of understanding, the present disclosure provides an implementation example of an evaluation model, including an information processing network and at least one evaluation network; illustratively, Figure 2The structural schematic diagram of an evaluation model shown includes an information processing network and N evaluation networks (evaluation network 1 to evaluation network N); different evaluation networks correspond to different evaluation dimensions. The information processing network is used to fuse the description information samples and multimedia samples corresponding to the preset generation model received by the evaluation model to obtain sample fusion information corresponding to the preset generation model. Specifically, the information processing network can respectively extract the first feature corresponding to the description information sample and the second feature corresponding to the multimedia sample, and then fuse the first feature and the second feature to obtain sample fusion information. For each evaluation network, the evaluation network is used to output the evaluation result of the preset generation model under the evaluation dimension corresponding to the evaluation network according to the sample fusion information. Figure 2 For example only, N evaluation networks can share one information processing network, and N evaluation networks and one information processing network can be considered as an evaluation model that can evaluate N evaluation dimensions at the same time, or N evaluation networks and one information processing network can be considered as evaluation models corresponding to N different evaluation dimensions, without limitation here. In practical applications, the evaluation model can be trained based on training samples with evaluation result labels corresponding to each preset evaluation dimension. For example, the evaluation result label can be manually set by a professional evaluator or user, so the trained evaluation model can output network evaluation results that are close to manual feedback, which can be more in line with the user's perception and ensure the accuracy of the evaluation.

[0044] Through the above method, the evaluation result of each preset generation model under at least one preset evaluation dimension can be obtained conveniently and reliably through the evaluation model.

[0045] The present disclosure further provides a specific implementation example of the above step B, which can be implemented by referring to the following steps B1 and B2:

[0046] Step B1, respectively obtain the similarity between the target description information and a plurality of preset description information samples, so as to screen out target samples matching the target description information from the plurality of description information samples based on the similarity acquisition result; wherein the number of target samples satisfies a preset number condition. The preset number condition may be, for example, that X target samples need to be obtained, that is, the number of target samples to be obtained is clearly given. Alternatively, the preset number condition may also be that X to Y target samples need to be obtained, that is, only a range of the number of target samples to be obtained is given, and any target sample within this range is acceptable.

[0047] Exemplarily, the similarity between the target sample and the target description information is not lower than the similarity between the target description information and the remaining description information samples except the target sample in the multiple description information samples. Taking the example that the number of target samples indicates that X target samples need to be obtained, the target sample is the first X description information samples with the highest similarity among the multiple description information samples. For example, if the target description information is "a little boy with a cat", the first few target samples selected that are most similar to the target description information may be "a little girl with a cat", "a little boy with a dog", etc.

[0048] Step B2: Based on the target sample and model information of the plurality of preset generation models, a plurality of candidate generation models are selected from the plurality of preset generation models.

[0049] In some specific examples, the model information of the preset generation model includes the evaluation result of the preset generation model under at least one preset evaluation dimension based on the description information sample corresponding to the preset generation model; on this basis, based on the target sample and the model information of multiple preset generation models, multiple candidate generation models are selected from multiple preset generation models, including: for each preset evaluation dimension, based on each target sample and the model information of multiple preset generation models, the preset generation model with the best evaluation result corresponding to each target sample under the preset evaluation dimension is determined as the candidate generation model. That is, the model with the best evaluation result (such as the highest score) corresponding to each preset evaluation dimension is found, so that these models with better performance under each evaluation dimension are used as candidate generation models, so that the language model can further screen out the most suitable target generation model from them.

[0050] Afterwards, the language model can perform task analysis on the target description information, determine the target evaluation dimension that the target description information focuses on, and then determine the target generation model based on the candidate generation model under the evaluation dimension. If there are multiple candidate generation models under the evaluation dimension, these candidate generation models can be further compared, such as comparing the similarity between their corresponding target samples and the target description information, comparing their model structures, required operating resources, etc., and finally selecting a candidate generation model as the target generation model.

[0051] Based on the foregoing, the present disclosure provides the following embodiments: Figure 3 Schematic diagram of the selection process of a generative model shown.

[0052] First, two parts are illustrated. In the first part, the method of obtaining the model information of the generation model is mainly illustrated: specifically, one or more description information samples are input into multiple generation models (generation model 1 to generation model M) to obtain multimedia samples output by M generation models for the description information samples, and then evaluated based on N evaluation dimensions (evaluation dimension 1 to evaluation dimension N), and the model information of the M generation models is determined based on the evaluation results. In the second part, the model selection method is mainly illustrated: the target description information and the model information of the generation model are input into the language model so that the language model selects the target generation model from the M generation models. In some specific implementation examples, in order to improve the model selection efficiency and model selection accuracy, the candidate generation model can be first determined based on the model information of the M generation models and the target description information, and the model information of the candidate generation model is provided to the language model. In addition, the model information of the M generation models can also be directly provided to the language model, so that the language model can directly select the target generation model that best suits the target description information based on the model information of the M generation models, which is not limited here.

[0053] Based on the foregoing, the present disclosure provides the following embodiments: Figure 4 The flowchart of a method for generating multimedia information shown in FIG. 1 mainly includes the following steps S402 to S410:

[0054] Step S402: upon receiving target description information of the multimedia information to be generated, obtaining model information of a plurality of preset generation models.

[0055] Step S404: Select multiple candidate generation models from the multiple preset generation models based on the target description information and the model information of the multiple preset generation models.

[0056] Step S406, obtaining target prompt information based on the target description information and model information of multiple candidate generation models.

[0057] Step S408: Based on the target prompt information, a target generation model is selected from a plurality of candidate generation models using a preset language model.

[0058] Step S410: generating target multimedia information corresponding to the target description information through a target generation model.

[0059] The specific implementation of the above steps can refer to the above related content, which will not be repeated here. Compared with the method of directly inputting the target description information into the specified generation model when receiving the target description information in the related art, Figure 4The above-mentioned method provided can first preliminarily select a candidate generation model from multiple preset generation models based on the target description information and the model information of multiple preset generation models, and then combine the target description information and the model information of multiple candidate generation models into target prompt information corresponding to the language model to prompt the language model to select a target generation model from multiple candidate generation models. With the help of the strong information processing capability of the language model, the most suitable target generation model is selected according to the model information of multiple candidate generation models to generate multimedia information corresponding to the current target description information, which effectively ensures the reliability of the multimedia information and also helps to give full play to the advantages of generation models in various fields.

[0060] Corresponding to the aforementioned method for selecting a generation model, the present disclosure provides a device for selecting a generation model. Figure 5 Schematic diagram of a structure of a selection device for generating a model provided by an embodiment of the present disclosure. The device can be implemented by software and / or hardware and can generally be integrated in an electronic device, such as Figure 5 As shown, the selection device for generating the model includes:

[0061] The information acquisition module 502 is used to acquire target prompt information; wherein the target prompt information includes target description information of the multimedia information to be generated and model information of multiple candidate generation models; the candidate generation model is a model with the ability to generate multimedia information, and the model information includes the evaluation result of the candidate generation model corresponding to at least one preset evaluation dimension;

[0062] The model selection module 504 is used to select a target generation model from multiple candidate generation models based on the target prompt information through a preset language model; wherein the target generation model is used to generate target multimedia information corresponding to the target description information.

[0063] Compared with directly using a specified generation model to generate multimedia information corresponding to the target description information, the above-mentioned device provided by the embodiment of the present disclosure can flexibly and reliably select a more suitable generation model, thereby effectively ensuring the generation effect of the multimedia information and improving the user experience.

[0064] In some embodiments, the information acquisition module 502 is specifically used to: upon receiving target description information of the multimedia information to be generated, obtain model information of multiple preset generation models; based on the target description information and the model information of the multiple preset generation models, select multiple candidate generation models from the multiple preset generation models; based on the target description information and the model information of the multiple candidate generation models, obtain target prompt information.

[0065] In some embodiments, a model information acquisition module is also included, which is used to obtain multimedia samples output by the preset generation model for the description information samples corresponding to the preset generation model; wherein, one preset generation model corresponds to one or more description information samples, and the description information samples corresponding to different preset generation models are the same or different; based on the multimedia samples output by the preset generation model for its corresponding description information samples, an evaluation result of the preset generation model under at least one preset evaluation dimension is obtained.

[0066] In some embodiments, the model information acquisition module is also used to input the description information samples corresponding to the preset generation model, and the multimedia samples output by the preset generation model for their corresponding description information samples, into a preset evaluation model, so as to output the evaluation results of the preset generation model under at least one preset evaluation dimension through the evaluation model.

[0067] In some embodiments, the evaluation model includes an information processing network and at least one evaluation network; different evaluation networks correspond to different evaluation dimensions; the information processing network is used to fuse the description information samples and multimedia samples corresponding to the preset generation model received by the evaluation model to obtain sample fusion information corresponding to the preset generation model; for each evaluation network, the evaluation network is used to output the evaluation result of the preset generation model under the evaluation dimension corresponding to the evaluation network according to the sample fusion information.

[0068] In some embodiments, the information acquisition module 502 is specifically used to: respectively obtain the similarity between the target description information and multiple preset description information samples, so as to filter out target samples matching the target description information from the multiple description information samples based on the similarity acquisition result; wherein the number of target samples satisfies a preset number condition; based on the target samples and the model information of the multiple preset generation models, select multiple candidate generation models from the multiple preset generation models.

[0069] In some implementations, the similarity between the target sample and the target description information is not lower than the similarity between the target description information and the remaining description information samples among the multiple description information samples except the target sample.

[0070] In some embodiments, the model information of the preset generation model includes an evaluation result of the preset generation model under at least one preset evaluation dimension obtained based on the descriptive information sample corresponding to the preset generation model; the information acquisition module 502 is specifically used to: for each preset evaluation dimension, based on each of the target samples and the model information of the multiple preset generation models, determine the preset generation model with the best evaluation result corresponding to each of the target samples under the preset evaluation dimension as a candidate generation model.

[0071] In some implementations, the model selection module 504 is specifically used to: perform task analysis on the target description information to obtain analysis results; and select a target generation model from the multiple candidate generation models based on the analysis results and model information of the multiple candidate generation models.

[0072] The generation model selection device provided in the embodiments of the present disclosure can execute the generation model selection method provided in any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0073] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described device embodiment can refer to the corresponding process in the method embodiment, and will not be repeated here.

[0074] An embodiment of the present disclosure provides an electronic device, which includes: a storage device on which a computer program is stored; and a processing device for executing the computer program in the storage device to implement the steps of any method in the present disclosure.

[0075] Reference below Figure 6 , which shows a schematic diagram of the structure of an electronic device 600 suitable for implementing the embodiment of the present disclosure. The terminal device in the embodiment of the present disclosure may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 6 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0076] like Figure 6As shown, the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0077] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 6 The electronic device 600 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0078] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 609, or installed from a storage device 608, or installed from a ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0079] In addition to the above-mentioned methods and devices, the embodiments of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, cause the processor to perform the image processing method provided by the embodiments of the present disclosure. The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present disclosure, the programming languages ​​including object-oriented programming languages ​​such as Java, C++, etc., and also conventional procedural programming languages ​​such as "C" language or similar programming languages. The program code may be executed entirely on a user computing device, partially on a user device, as an independent software package, partially on a user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0080] In addition, the embodiment of the present disclosure may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the method for selecting a generation model provided by the embodiment of the present disclosure.

[0081] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0082] The embodiment of the present disclosure also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the method for selecting a generation model in the embodiment of the present disclosure.

[0083] It is understandable that before using the technical solutions disclosed in the various embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0084] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0085] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0086] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0087] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0088] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for selecting a generative model, characterized in that: include: Obtaining target prompt information; wherein the target prompt information includes target description information of the multimedia information to be generated and model information of multiple candidate generation models; the candidate generation model is a model with the ability to generate the multimedia information, and the model information includes an evaluation result of the candidate generation model under at least one preset evaluation dimension; Based on the target prompt information, a target generation model is selected from the multiple candidate generation models through a preset language model; wherein the target generation model is used to generate target multimedia information corresponding to the target description information.

2. The method according to claim 1, characterized in that The obtaining of target prompt information includes: Upon receiving target description information of the multimedia information to be generated, obtaining model information of a plurality of preset generation models; Based on the target description information and the model information of the plurality of preset generation models, selecting a plurality of candidate generation models from the plurality of preset generation models; Target prompt information is obtained based on the target description information and the model information of the multiple candidate generation models.

3. The method according to claim 2, characterized in that The model information of the preset generation model is obtained in advance through the following steps: Obtaining a multimedia sample output by the preset generation model for a description information sample corresponding to the preset generation model; wherein one preset generation model corresponds to one or more description information samples, and description information samples corresponding to different preset generation models are the same or different; Based on the multimedia sample output by the preset generation model for its corresponding description information sample, an evaluation result of the preset generation model under at least one preset evaluation dimension is obtained.

4. The method according to claim 3, characterized in that The step of obtaining an evaluation result of the preset generation model under at least one preset evaluation dimension based on the multimedia sample output by the preset generation model for the corresponding description information sample comprises: The description information samples corresponding to the preset generation model and the multimedia samples output by the preset generation model for the corresponding description information samples are both input into a preset evaluation model, so as to output the evaluation result of the preset generation model under at least one preset evaluation dimension through the evaluation model.

5. The method according to claim 4, characterized in that The evaluation model includes an information processing network and at least one evaluation network; different evaluation networks correspond to different evaluation dimensions; The information processing network is used to fuse the description information samples and multimedia samples corresponding to the preset generation model received by the evaluation model to obtain sample fusion information corresponding to the preset generation model; For each of the evaluation networks, the evaluation network is used to output an evaluation result of the preset generation model under the evaluation dimension corresponding to the evaluation network according to the sample fusion information.

6. The method according to claim 2, characterized in that The selecting a plurality of candidate generation models from the plurality of preset generation models based on the target description information and the model information of the plurality of preset generation models comprises: Respectively obtaining similarities between the target description information and a plurality of preset description information samples, so as to filter out target samples matching the target description information from the plurality of description information samples based on the similarity obtaining result; wherein the number of the target samples satisfies a preset number condition; Based on the target sample and the model information of the multiple preset generation models, multiple candidate generation models are selected from the multiple preset generation models.

7. The method according to claim 6, characterized in that The similarity between the target sample and the target description information is not lower than the similarity between the target description information and the remaining description information samples among the multiple description information samples except the target sample.

8. The method according to claim 6, characterized in that The model information of the preset generation model includes an evaluation result obtained by the preset generation model under at least one preset evaluation dimension based on a description information sample corresponding to the preset generation model; The selecting a plurality of candidate generation models from the plurality of preset generation models based on the target sample and the model information of the plurality of preset generation models comprises: For each preset evaluation dimension, based on each of the target samples and the model information of the multiple preset generation models, the preset generation model with the best evaluation result corresponding to each of the target samples under the preset evaluation dimension is determined as the candidate generation model.

9. The method according to claim 1, characterized in that: The step of selecting a target generation model from the plurality of candidate generation models comprises: Performing task analysis on the target description information to obtain an analysis result; Based on the parsing result and the model information of the multiple candidate generation models, a target generation model is selected from the multiple candidate generation models.

10. A device for selecting a generation model, characterized in that: include: An information acquisition module, used to acquire target prompt information; wherein the target prompt information includes target description information of the multimedia information to be generated and model information of multiple candidate generation models; the candidate generation model is a model with the ability to generate the multimedia information, and the model information includes an evaluation result of the candidate generation model corresponding to at least one preset evaluation dimension; A model selection module is used to select a target generation model from the multiple candidate generation models based on the target prompt information through a preset language model; wherein the target generation model is used to generate target multimedia information corresponding to the target description information.

11. An electronic device, characterized in that: The electronic device comprises: a storage device having a computer program stored thereon; A processing device is used to execute the computer program in the storage device to implement the steps of the method for selecting a generation model according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and the computer program is used to execute the method for selecting a generation model as described in any one of claims 1 to 9.

13. A computer program product, characterized in that The invention comprises a computer program, which, when executed by a processor, implements the method for selecting a generation model according to any one of claims 1 to 9.

Images