Multi-dimensional fine-grained reward method and apparatus for image and video generation

Abstract

This invention relates to the fields of computer vision and multimodal generation technology, and particularly to a multidimensional fine-grained reward method and apparatus for image and video generation. The method includes: constructing an evaluation system for each target image or video, and extracting the binary judgment result for each image or video from the evaluation system; training a pre-constructed visual language foundation model using each image or video, its multidimensional multiple-choice questions, and the binary judgment result to generate a multidimensional fine-grained reward model; inputting the image or video to be tested into the multidimensional fine-grained reward model to generate actual binary judgment results; and linearly weighting the actual binary judgment results to predict the visual preference result of the image or video to be tested. This solves the problems of existing human feedback reinforcement learning methods that over-optimize or weaken certain factors, leading to unsatisfactory optimized results.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and multimodal generation technology, particularly to the field of human feedback alignment technology for multimodal generation, and especially to a multidimensional fine-grained reward method and apparatus for image and video generation. Background Technology

[0002] Visual generative models, including text-to-image and text-to-video generation, have seen rapid development recently. Based on textual descriptions, these visual generative models can generate high-quality images or videos. Inspired by Human Feedback Reinforcement Learning (RLHF) in Large Language Models (LLMs), researchers have found that aligning text to images according to human preferences is crucial due to various challenges such as alignment, fidelity, aesthetics, and security. Works such as ImageReward have developed diffusion model optimization methods based on these reward models or preference datasets. With the development of text-to-video models, similar evaluation and optimization methods have also been tried and improved.

[0003] Despite progress, current RLHF methods for text-to-vision models still face significant challenges. First, reward models are biased and uninterpretable; current reward models learn from human preferences, which involve numerous trade-offs between different factors, leading to preference bias. Second, evaluating the dynamic quality of videos is difficult. Third, existing RLHF methods tend to over-optimize or weaken certain factors, resulting in suboptimal optimized results. Summary of the Invention

[0004] This invention provides a multi-dimensional fine-grained reward method and apparatus for image and video generation, in order to solve the problems of existing human feedback reinforcement learning methods that over-optimize or weaken certain factors, resulting in unsatisfactory optimized results.

[0005] A first aspect of the present invention provides a multi-dimensional fine-grained reward method for image and video generation, comprising the following steps: constructing an evaluation system for each target image or video, and extracting the binary judgment result of each image or video from the evaluation system; training a pre-constructed visual language base model using each image or video, the multi-dimensional multiple-choice questions of each image or video, and the binary judgment result of each image or video to generate a multi-dimensional fine-grained reward model; inputting the image or video to be tested into the multi-dimensional fine-grained reward model to generate an actual binary judgment result; and linearly weighting the actual binary judgment result to predict the visual preference result of the image or video to be tested.

[0006] Optionally, the step of constructing an evaluation system for each target image or video, and extracting a binary judgment result from the evaluation system for each image or video, includes:

[0007] Each target image or video and its corresponding multi-dimensional multiple-choice question are sent to the target annotator, and the target annotator is instructed to annotate each target image or video according to the pre-designed multi-dimensional multiple-choice question;

[0008] After the target annotator has completed the annotation, the options of the multi-dimensional multiple-choice question are decomposed into multiple judgment questions to form an evaluation system for each image or video;

[0009] Extract the binary judgment result for each image or video from the evaluation system for each image or video.

[0010] Optionally, the step of training a pre-built visual language foundation model using each image or video, the multi-dimensional multiple-choice questions of each image or video, and the binary judgment results of each image or video to generate a multi-dimensional fine-grained reward model includes:

[0011] Balanced sampling is performed on each image or video, the multi-dimensional multiple-choice questions of each image or video, and the binary judgment results of each image or video to obtain a balanced sampling training dataset;

[0012] The visual language base model is fine-tuned using the balanced sampling training dataset to generate the multi-dimensional fine-grained reward model.

[0013] Optionally, the step of linearly weighting the actual binary judgment result to predict the visual preference result of the image or video to be tested includes:

[0014] The actual binary judgment result is mapped to a preset feature value, and the preset feature value is linearly weighted to obtain a weighted sum result;

[0015] The weighted sum results are categorized into preset category labels based on a pre-constructed human preference dataset;

[0016] Linear regression is performed using the weighted sum and the preset category labels to predict the visual preference results of the image or video to be tested.

[0017] A second aspect of the present invention provides a multi-dimensional fine-grained reward device for image and video generation, comprising: a construction and extraction module for constructing an evaluation system for each target image or video and extracting a binary judgment result for each image or video from the evaluation system; a training module for training a pre-constructed visual language base model using each image or video, the multi-dimensional multiple-choice questions of each image or video, and the binary judgment result of each image or video to generate a multi-dimensional fine-grained reward model; a generation module for inputting a test image or video into the multi-dimensional fine-grained reward model to generate an actual binary judgment result; and a prediction module for linearly weighting the actual binary judgment result to predict the visual preference result of the test image or video.

[0018] Optionally, the construction extraction module includes:

[0019] The annotation unit is used to send each target image or video and its corresponding multi-dimensional multiple-choice questions to the target annotator, and instruct the target annotator to annotate each target image or video according to the pre-designed multi-dimensional multiple-choice questions;

[0020] The decomposition unit is used to decompose the options of the multi-dimensional multiple-choice question into multiple judgment questions after the target annotator has completed the annotation, so as to form an evaluation system for each image or video.

[0021] An extraction unit is used to extract the binary judgment result of each image or video from the evaluation system of each image or video.

[0022] Optionally, the training module includes:

[0023] The sampling unit is used to perform balanced sampling on each image or video, the multi-dimensional multiple-choice questions of each image or video, and the binary judgment results of each image or video to obtain a balanced sampling training dataset.

[0024] The fine-tuning unit is used to fine-tune the visual language base model using the balanced sampling training dataset to generate the multi-dimensional fine-grained reward model.

[0025] Optionally, the prediction module includes:

[0026] The weighting unit is used to map the actual binary judgment result to a preset feature value, and to linearly weight the preset feature value to obtain a weighted sum result;

[0027] A classification unit is used to classify the weighted sum result into a preset category label based on a pre-built human preference dataset;

[0028] The prediction unit is used to perform linear regression using the weighted sum result and the preset category label to predict the visual preference result of the image or video to be tested.

[0029] A third aspect of the present invention provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the multi-dimensional fine-grained reward method for image and video generation as described in the above embodiments.

[0030] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described multi-dimensional fine-grained reward method for image and video generation.

[0031] The multi-dimensional fine-grained reward method and apparatus for image and video generation proposed in this invention constructs a multi-objective fine-grained multimodal evaluation system and reward model, thereby providing accurate evaluation results. Based on the accurate evaluation results, a multi-objective optimization algorithm can be developed to stably adjust the visual generation model, avoid over-optimization or lack of optimization for certain factors, and significantly improve the multimodal generation effect.

[0032] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0033] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0034] Figure 1 A flowchart illustrating a multi-dimensional fine-grained reward method for image and video generation provided in an embodiment of the present invention;

[0035] Figure 2 This is a flowchart illustrating the workflow of a multi-dimensional fine-grained reward method for image and video generation provided in an embodiment of the present invention.

[0036] Figure 3 A block diagram illustrating a multi-dimensional fine-grained reward device for image and video generation provided in an embodiment of the present invention;

[0037] Figure 4 This is a block diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0038] The following describes embodiments of the present invention in detail, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and are not to be construed as limiting the present invention.

[0039] The following description, with reference to the accompanying drawings, illustrates a multi-dimensional fine-grained reward method and apparatus for image and video generation according to embodiments of the present invention.

[0040] Figure 1 This is a flowchart illustrating a multi-dimensional fine-grained reward method for image and video generation provided in an embodiment of the present invention.

[0041] like Figure 1 As shown, the multi-dimensional fine-grained reward method for image and video generation includes the following steps:

[0042] In step S101, an evaluation system for each target image or video is constructed, and a binary judgment result for each image or video is extracted from the evaluation system for each image or video.

[0043] In some embodiments, an evaluation system is constructed for each target image or video, and a binary judgment result is extracted from the evaluation system of each image or video, including:

[0044] Send each target image or video and its corresponding multi-dimensional multiple-choice questions to the target annotator, and instruct the target annotator to annotate each target image or video according to the pre-designed multi-dimensional multiple-choice questions;

[0045] After the target annotator has completed the annotation, the options of the multi-dimensional multiple-choice questions are broken down into multiple judgment questions to form an evaluation system for each image or video;

[0046] Extract the binary judgment result for each image or video from the evaluation system for each image or video.

[0047] In actual implementation, such as Figure 2 As shown, human preferences when evaluating images or videos are often the result of the interaction of multiple factors, thus requiring a balance among various considerations. To systematically deconstruct human preferences, embodiments of this invention develop a fine-grained, multi-dimensional preference decomposition framework.

[0048] For each target image or video, pre-design multiple-choice questions for each dimension. Send each target image or video and its corresponding multiple-choice questions to the target annotator and instruct the target annotator to annotate each target image or video according to the pre-designed multiple-choice questions. The annotation can include 3 million questions for 48k images and 2 million questions for 33k videos.

[0049] Furthermore, based on a fine-grained, multi-dimensional preference decomposition framework, during the annotation process, the annotator selects the option that best suits the image. After the annotator completes the annotation, the options of the multi-dimensional multiple-choice questions are decomposed into multiple judgment (yes / no) questions, ultimately forming an evaluation system for evaluating each image or video, which can be used for subsequent evaluation and reward model training.

[0050] It should be noted that, in order to address the challenges of video evaluation, a broad observation of the video's dynamic content, such as motion stability or motion quality, can be incorporated into the multiple-choice questions.

[0051] In step S102, the pre-constructed visual language base model is trained using each image or video, the multi-dimensional multiple-choice questions for each image or video, and the binary judgment results for each image or video to generate a multi-dimensional fine-grained reward model.

[0052] In some embodiments, a pre-built visual language foundation model is trained using each image or video, the multi-dimensional multiple-choice questions for each image or video, and the binary judgment result for each image or video to generate a multi-dimensional fine-grained reward model, including:

[0053] Balanced sampling is performed on each image or video, the multi-dimensional multiple-choice questions for each image or video, and the binary judgment results for each image or video to obtain the balanced sampling training dataset;

[0054] The visual language pedigree model is fine-tuned using a balanced sampling training dataset to generate a multi-dimensional, fine-grained reward model.

[0055] In actual implementation, such as Figure 2 As shown, by extracting the binary judgment result of each image or video from the evaluation system of each image or video, millions of labeled binary judgment results can be obtained. Then, balanced sampling is performed on each image or video, each multi-dimensional multiple-choice question of each image or video, and each binary judgment result of each image or video to solve the imbalance problem between positive and negative instances, ensuring that the number of positive and negative instances related to each judgment question is approximately equal, resulting in a balanced instruction composed of judgment questions (i.e., the training dataset after balanced sampling). Finally, the pre-built visual language base model is fine-tuned using the balanced instruction composed of judgment questions to generate the multi-dimensional fine-grained reward model. CogVLM2 is used as the visual language base model for image understanding, and CogVLM2-Video is used as the visual language base model for video understanding.

[0056] In step S103, the image or video to be tested is input into a multi-dimensional fine-grained reward model to generate an actual binary judgment result.

[0057] In step S104, the actual binary judgment results are linearly weighted to predict the visual preference results of the image or video to be tested.

[0058] In some embodiments, the actual binary judgment results are linearly weighted to predict the visual preference results of the image or video to be tested, including:

[0059] The actual binary judgment result is mapped to a preset feature value, and the preset feature value is linearly weighted to obtain a weighted sum result;

[0060] The weighted sum results are categorized into preset category labels based on a pre-built dataset of human preferences.

[0061] Linear regression is performed using weighted sums and predefined category labels to predict visual preference outcomes for the image or video being tested.

[0062] In actual implementation, such as Figure 2 As shown, the image or video to be tested is input into a multi-dimensional fine-grained reward model to generate a series of binary (yes or no) answers (i.e., the actual binary judgment results). The series of binary (yes or no) answers is mapped to preset feature values, such as mapping "yes" and "no" to "1" and "-1" respectively, and a feature vector is constructed. A set of linear weights is used to linearly weight the feature vector to predict human preferences, as shown in the following figure:

[0063] score = x T w = x1w1 + x2w2 + ... + x n w n

[0064] Where, x1...... x n For the eigenvectors, w1...... w n It is a set of linear weights.

[0065] To learn these parameters, embodiments of the present invention pre-collect a human preference dataset, calculate the feature difference for each pair of data, and assign corresponding labels based on the human preference dataset. Then, a logistic regression model is used to perform regression to predict the visual preference outcome for the image or video to be tested. The visual preference outcome (probability) is given by the following formula:

[0066]

[0067] The objective function to be minimized during the prediction process is defined as:

[0068] Loss(w)=-Ε[ylog(σ(Δx T w))+(1-y)log(1-σ(Δx Tw))]

[0069] Where Ε is the expected value, y is the preference label, and σ is the sigmoid function.

[0070] In addition, such as Figure 2 As shown, to ensure that all dimensions are reasonably boosted, the problem of multi-dimensional fine-grained reward methods for image and video generation is incorporated into these dimensions and formulated as rewards. Given two images, a dominance pair is defined to ensure that each dimension is not weakened in the preference.

[0071]

[0072] Where R() is the reward model, For evaluation dimensions, k is the dimension number.

[0073] In summary, the multi-dimensional fine-grained reward method for image and video generation proposed in this embodiment of the invention has the following advantages:

[0074] Beneficial effects:

[0075] (1) By designing a unified annotation system for image and video generation, the factors influencing human preferences were decomposed;

[0076] 2) It has high accuracy and interpretability in predicting human preferences, far superior to existing preference prediction methods, especially in video evaluation, where it outperforms VideoScore by 17.2%.

[0077] 3) Visual preference results can be incorporated into the Multi-Objective Preference Optimization (MPO) algorithm to stably adjust the visual generation model and avoid over-optimization or lack of optimization for certain factors. Furthermore, models using MPO outperform those directly adjusted using manual annotation or other reward models.

[0078] Next, with reference to the accompanying drawings, a multi-dimensional fine-grained reward device for image and video generation according to an embodiment of the present invention is described.

[0079] Figure 3 This is a block diagram of a multi-dimensional fine-grained reward device for image and video generation according to an embodiment of the present invention.

[0080] like Figure 3 As shown, the multi-dimensional fine-grained reward device 30 for image and video generation includes: a construction extraction module 301, a training module 302, a generation module 303, and a prediction module 304.

[0081] The extraction module 301 constructs an evaluation system for each target image or video and extracts the binary judgment result for each image or video from the evaluation system. The training module 302 trains a pre-constructed visual language foundation model using each image or video, its multi-dimensional multiple-choice questions, and the binary judgment result to generate a multi-dimensional fine-grained reward model. The generation module 303 inputs the image or video to be tested into the multi-dimensional fine-grained reward model to generate the actual binary judgment result. The prediction module 304 linearly weights the actual binary judgment result to predict the visual preference result of the image or video to be tested.

[0082] In some embodiments, the extraction module 301 includes:

[0083] The annotation unit is used to send each target image or video and its corresponding multi-dimensional multiple-choice questions to the target annotator, and instruct the target annotator to annotate each target image or video according to the pre-designed multi-dimensional multiple-choice questions;

[0084] The decomposition unit is used to break down the options of a multi-dimensional multiple-choice question into multiple judgment questions after the target annotator has completed the annotation, so as to form an evaluation system for each image or video.

[0085] The extraction unit is used to extract the binary judgment result of each image or video in the evaluation system of each image or video.

[0086] In some embodiments, the training module 302 includes:

[0087] The sampling unit is used to perform balanced sampling on each image or video, the multi-dimensional multiple-choice questions of each image or video, and the binary judgment results of each image or video to obtain the balanced sampling training dataset.

[0088] The fine-tuning unit is used to fine-tune the visual language base model using the balanced sampled training dataset to generate a multi-dimensional, fine-grained reward model.

[0089] In some embodiments, the prediction module 304 includes:

[0090] The weighting unit is used to map the actual binary judgment result to a preset feature value, and to linearly weight the preset feature value to obtain a weighted sum result;

[0091] The classification unit is used to classify the weighted sum results into preset category labels based on a pre-built dataset of human preferences;

[0092] The prediction unit is used to perform linear regression using the weighted sum and preset category labels to predict the visual preference results of the image or video to be tested.

[0093] It should be noted that the foregoing explanation of the multi-dimensional fine-grained reward method embodiment for image and video generation also applies to the multi-dimensional fine-grained reward device for image and video generation in this embodiment, and will not be repeated here.

[0094] The multi-dimensional fine-grained reward device for image and video generation proposed in the embodiments of the present invention constructs a multi-objective fine-grained multimodal evaluation system and reward model, thereby providing accurate evaluation results. Based on the accurate evaluation results, a multi-objective optimization algorithm can be developed to stably adjust the visual generation model, avoid over-optimization or lack of optimization for certain factors, and significantly improve the multimodal generation effect.

[0095] Figure 4 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. The electronic device may include:

[0096] The memory 401, the processor 402, and the computer program stored on the memory 401 and capable of running on the processor 402.

[0097] When the processor 402 executes the program, it implements the multi-dimensional fine-grained reward method for image and video generation provided in the above embodiments.

[0098] Furthermore, electronic devices also include:

[0099] Communication interface 403 is used for communication between memory 401 and processor 402.

[0100] The memory 401 is used to store computer programs that can run on the processor 402.

[0101] The memory 401 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0102] If the memory 401, processor 402, and communication interface 403 are implemented independently, then the communication interface 403, memory 401, and processor 402 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 4Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0103] Optionally, in a specific implementation, if the memory 401, processor 402, and communication interface 403 are integrated on a single chip, then the memory 401, processor 402, and communication interface 403 can communicate with each other through an internal interface.

[0104] Processor 402 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.

[0105] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described multi-dimensional fine-grained reward method for image and video generation.

[0106] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0107] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0108] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0109] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0110] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0111] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0112] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0113] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

Claims

1. A multi-dimensional fine-grained reward method for image and video generation, characterized in that, Includes the following steps: Construct an evaluation system for each target image or video, and extract a binary judgment result for each image or video from the evaluation system, specifically including: Each target image or video and its corresponding multi-dimensional multiple-choice question are sent to the target annotator, and the target annotator is instructed to annotate each target image or video according to the pre-designed multi-dimensional multiple-choice question; After the target annotator has completed the annotation, the options of the multi-dimensional multiple-choice question are decomposed into multiple judgment questions to form an evaluation system for each image or video; Extract the binary judgment result of each image or video from the evaluation system of each image or video; The pre-constructed visual language foundation model is trained using each image or video, the multi-dimensional multiple-choice questions in each image or video, and the binary judgment results of each image or video to generate a multi-dimensional fine-grained reward model, specifically including: Balanced sampling is performed on each image or video, the multi-dimensional multiple-choice questions of each image or video, and the binary judgment results of each image or video to obtain a balanced sampling training dataset; The visual language base model is fine-tuned using the balanced sampling training dataset to generate the multi-dimensional fine-grained reward model. The image or video to be tested is input into the multi-dimensional fine-grained reward model to generate the actual binary judgment result; The actual binary judgment results are linearly weighted to predict the visual preference results of the image or video to be tested, specifically including: The actual binary judgment result is mapped to a preset feature value, and the preset feature value is linearly weighted to obtain a weighted sum result; The weighted sum results are categorized into preset category labels based on a pre-constructed human preference dataset; Linear regression is performed using the weighted sum and the preset category labels to predict the visual preference results of the image or video to be tested.

2. A multi-dimensional fine-grained reward device for image and video generation, characterized in that, include: An extraction module is constructed to build an evaluation system for each target image or video, and to extract the binary judgment result of each image or video from the evaluation system. Specifically, this includes: The annotation unit is used to send each target image or video and its corresponding multi-dimensional multiple-choice questions to the target annotator, and instruct the target annotator to annotate each target image or video according to the pre-designed multi-dimensional multiple-choice questions; The decomposition unit is used to decompose the options of the multi-dimensional multiple-choice question into multiple judgment questions after the target annotator has completed the annotation, so as to form an evaluation system for each image or video. An extraction unit is used to extract the binary judgment result of each image or video from the evaluation system of each image or video; The training module is used to train a pre-built visual language foundation model using each image or video, the multi-dimensional multiple-choice questions of each image or video, and the binary judgment results of each image or video, to generate a multi-dimensional fine-grained reward model, specifically including: The sampling unit is used to perform balanced sampling on each image or video, the multi-dimensional multiple-choice questions of each image or video, and the binary judgment results of each image or video to obtain a balanced sampling training dataset. The fine-tuning unit is used to fine-tune the visual language base model using the balanced sampling training dataset to generate the multi-dimensional fine-grained reward model. The generation module is used to input the image or video to be tested into the multi-dimensional fine-grained reward model to generate the actual binary judgment result; The prediction module is used to linearly weight the actual binary judgment results to predict the visual preference results of the image or video to be tested, specifically including: The weighting unit is used to map the actual binary judgment result to a preset feature value, and to linearly weight the preset feature value to obtain a weighted sum result; A classification unit is used to classify the weighted sum result into a preset category label based on a pre-built human preference dataset; The prediction unit is used to perform linear regression using the weighted sum result and the preset category label to predict the visual preference result of the image or video to be tested.

3. An electronic device, characterized in that, include: The memory, the processor, and the computer program stored in the memory and executable on the processor, the processor executing the program to implement the multi-dimensional fine-grained reward method for image and video generation as described in claim 1.

4. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the multi-dimensional fine-grained reward method for image and video generation as described in claim 1.

Images