Stream matching image generation method and device based on online reinforcement learning optimization

Abstract

The invention relates to the technical field of artificial intelligence, the scheme can be applied to the fields of finance and medical treatment, and the invention provides a stream matching image generation method and device based on online reinforcement learning optimization, and the method comprises the steps: converting an input prompt word into a plurality of initial noises through a stream matching image generation model; in the stream matching image generation model, based on an SDE sampling strategy, combining a predefined strategy parameter and according to a preset first sampling time step, carrying out step-by-step denoising on the plurality of initial noises, and generating a plurality of candidate images; obtaining an advantage value corresponding to each candidate image through standard deviation normalization; introducing a GRPO reinforcement learning mechanism, and optimizing the strategy parameter based on the advantage value corresponding to each candidate image to obtain an optimized strategy parameter; and based on the optimized strategy parameters, generating a high-quality image corresponding to the cue word through a stream matching image generation model. According to the method, the accuracy of combined image generation and character rendering tasks is improved, and the training time is shortened.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and the solution can be applied to the fields of finance and healthcare. In particular, it relates to a method and apparatus for generating stream matching images based on online reinforcement learning optimization. Background Technology

[0002] In recent years, image generation models, especially those based on diffusion or flow matching, have achieved remarkable results in text-to-image (T2I) tasks. Flow matching models have gradually become mainstream due to their continuous-time generation characteristics, efficient inference, and good training stability. However, when dealing with complex combined images (containing multiple objects, fine attributes, and spatial relationships) and text rendering tasks, existing flow matching models have the following shortcomings:

[0003] First, traditional models rely on deterministic generation paths and reason using ordinary differential equations (ODEs). The process lacks randomness, which improves generation efficiency but limits its application to reinforcement learning frameworks. It is difficult to explore effective strategies, resulting in insufficient sample diversity and poor task adaptability during training. For example, in medical image generation, diverse lesion morphology simulation is needed to assist diagnosis, and fixed paths are difficult to cover complex pathological manifestations. In financial scenarios, generating dynamic market visualization maps is difficult to adapt to the complex data relationships of a single model.

[0004] Secondly, in order to obtain high-quality images, existing models have many denoising steps and high data sampling computation costs. When reinforcement learning requires frequent sampling to update the strategy, this severely restricts training efficiency and practicality.

[0005] Third, when reinforcement learning is introduced to align with human preferences, traditional strategies rely on offline reinforcement learning or scoring regression, which lack online update capabilities and often require retraining of large-scale models, resulting in high engineering costs. More importantly, some methods are prone to "reward hijacking," which improves task scores but severely damages image quality and diversity, deviating from the requirements of high fidelity and richness for generative models.

[0006] Therefore, existing stream matching models suffer from poor task adaptability and high sampling computational overhead in text-to-image generation tasks. Summary of the Invention

[0007] This invention provides a method and apparatus for generating images by stream matching based on online reinforcement learning optimization, aiming to solve the problems of poor task adaptability and high sampling computation overhead in existing stream matching models for text-to-image tasks.

[0008] In a first aspect, embodiments of the present invention provide a stream matching image generation method based on online reinforcement learning optimization, the method comprising:

[0009] The input prompt words are converted into multiple initial noises using a stream matching image generation model;

[0010] In the stream matching image generation model, based on the SDE sampling strategy, combined with predefined strategy parameters, multiple initial noises are gradually denoised according to a preset first sampling time step to generate multiple candidate images.

[0011] In the stream matching image generation model, the advantage value corresponding to each candidate image is obtained by standard deviation normalization;

[0012] In the stream matching image generation model, the GRPO reinforcement learning mechanism is introduced to optimize the policy parameters based on the advantage value corresponding to each candidate image, so as to obtain the optimized policy parameters.

[0013] Based on the optimized strategy parameters, a high-quality image corresponding to the prompt word is generated through a stream matching image generation model.

[0014] Secondly, embodiments of the present invention also provide a stream matching image generation apparatus based on online reinforcement learning optimization, the apparatus comprising:

[0015] The transformation unit is used to convert the input prompt words into multiple initial noises using a stream matching image generation model;

[0016] The denoising unit is used in the stream matching image generation model to gradually denoise multiple initial noises based on the SDE sampling strategy and combined with predefined strategy parameters according to a preset first sampling time step, thereby generating multiple candidate images.

[0017] The acquisition unit is used to acquire the advantage value corresponding to each candidate image by standard deviation normalization in the stream matching image generation model;

[0018] The optimization unit is used to introduce the GRPO reinforcement learning mechanism in the stream matching image generation model, and optimize the policy parameters based on the advantage value corresponding to each candidate image to obtain the optimized policy parameters.

[0019] The generation unit is used to generate a high-quality image corresponding to the prompt word based on the optimized strategy parameters and through a stream matching image generation model.

[0020] Thirdly, embodiments of the present invention also provide an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect above.

[0021] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, the computer program including program instructions that, when executed by a processor, can implement the method described in the first aspect.

[0022] This invention provides a method and apparatus for stream matching image generation based on online reinforcement learning optimization. The method includes: converting an input prompt word into multiple initial noises using a stream matching image generation model; in the stream matching image generation model, based on the SDE sampling strategy and combined with predefined strategy parameters, progressively denoising the multiple initial noises according to a preset first sampling time step to generate multiple candidate images; in the stream matching image generation model, obtaining the advantage value corresponding to each candidate image through standard deviation normalization; in the stream matching image generation model, introducing the GRPO reinforcement learning mechanism, optimizing the strategy parameters based on the advantage value corresponding to each candidate image to obtain optimized strategy parameters; and based on the optimized strategy parameters, generating a high-quality image corresponding to the prompt word through the stream matching image generation model. In complex task performance, this invention significantly improves the accuracy of combined image generation and text rendering tasks by leveraging the GRPO reinforcement learning mechanism and combining the randomness of the SDE sampling strategy with optimized strategy parameters. In terms of training efficiency, using a small number of first sampling time steps for progressive denoising shortens the training time without compromising the final task performance. Attached Figure Description

[0023] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 A flowchart illustrating the stream matching image generation method based on online reinforcement learning optimization provided in an embodiment of the present invention;

[0025] Figure 2 A schematic block diagram of a stream matching image generation device based on online reinforcement learning optimization provided in an embodiment of the present invention;

[0026] Figure 3 A schematic block diagram of an electronic device provided in an embodiment of the present invention;

[0027] Figure 4 This is a schematic diagram illustrating the application environment of the online reinforcement learning-optimized stream matching image generation method provided in this embodiment of the invention. Detailed Implementation

[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0030] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0031] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations. Embodiments of this invention provide a method and apparatus for generating flow matching images based on online reinforcement learning optimization. For details on this method for generating flow matching images based on online reinforcement learning optimization, please refer to... Figure 4 , Figure 4 This diagram illustrates the application environment of the online reinforcement learning-optimized stream matching image generation method provided in this embodiment of the invention. The online reinforcement learning-optimized stream matching image generation method is applied in applications such as... Figure 4In this application environment, the user terminal communicates with the server via a network. The server uses a stream matching image generation model to convert the prompt words from the user terminal into multiple initial noises. In the stream matching image generation model, based on the SDE sampling strategy and combined with predefined strategy parameters, the initial noises are progressively denoised according to a preset first sampling time step to generate multiple candidate images. In the stream matching image generation model, the advantage value corresponding to each candidate image is obtained through standard deviation normalization. In the stream matching image generation model, the GRPO reinforcement learning mechanism is introduced to optimize the strategy parameters based on the advantage value corresponding to each candidate image, resulting in optimized strategy parameters. Based on the optimized strategy parameters, a high-quality image corresponding to the prompt word is generated through the stream matching image generation model. In terms of performance on complex tasks, this embodiment of the invention, by leveraging the GRPO reinforcement learning mechanism and combining the randomness of the SDE sampling strategy with the optimization of strategy parameters, significantly improves the accuracy of combined image generation and text rendering tasks. In terms of training efficiency, using a small number of first sampling time steps for progressive denoising shortens the training time without causing any loss in the final task performance. The user terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and other smart devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers. The invention will now be described in detail through specific embodiments.

[0032] Figure 1 This is a flowchart illustrating the stream matching image generation method based on online reinforcement learning optimization provided in an embodiment of the present invention. Figure 1 As shown, the method includes the following steps S110-S150.

[0033] S110. The input prompt words are converted into multiple initial noises using a flow matching image generation model.

[0034] In this embodiment, the input prompt word is converted into multiple initial noises using a flow matching image generation model. In the medical field, the input prompt word can be "chest CT image containing lung nodules and pleural thickening". In the financial field, the input prompt word can be "K-line chart of a stock's closing price and trading volume over the past year".

[0035] In one embodiment, step S110 includes: in the stream matching image generation model, generating multiple initial noises based on a standard Gaussian distribution for the input prompt word.

[0036] In this embodiment, in the stream matching image generation model, multiple initial noises are generated based on a standard Gaussian distribution for the input prompt words. This process leverages the randomness of the standard Gaussian distribution to provide diverse "starting points" for subsequent image generation.

[0037] S120. In the stream matching image generation model, based on the SDE sampling strategy and combined with predefined strategy parameters, multiple initial noises are gradually denoised according to a preset first sampling time step to generate multiple candidate images.

[0038] In this embodiment, in the stream matching image generation model, based on the SDE sampling strategy and combined with predefined strategy parameters, multiple initial noises are gradually denoised according to a preset first sampling time step to generate multiple candidate images; the initial noises correspond one-to-one with the candidate images.

[0039] This scheme is based on the SDE sampling strategy, which transforms the originally deterministic ODE (Ordinary Differential Equation) generation process into a stochastic differential equation (SDE) form with equivalent marginal distribution, thus making the model generation path more diverse. Secondly, in order to improve the training sampling efficiency, the sampling time step is significantly reduced from the traditional 40 steps to 10 steps (i.e., the first sampling time step) during the training phase, and multiple candidate images (i.e., low-cost samples) are generated based on the first sampling time step. Policy learning is performed using low-cost samples. Although the quality of the training samples is relatively low, it still provides sufficient task information during the policy update process.

[0040] In one embodiment, step S120 includes: in the stream matching image generation model, combining the SDE sampling strategy with the strategy parameters to transform it into a stochastic differential equation with strategy parameters; in the stream matching image generation model, according to the stochastic differential equation, progressively denoising the multiple initial noises according to the first sampling time step to generate multiple candidate images; wherein, in each of the first sampling time steps, Gaussian noise perturbation is introduced.

[0041] In this embodiment, in the stream matching image generation model, the SDE sampling strategy is combined with the strategy parameters to transform it into a stochastic differential equation with strategy parameters, in the form dx t =f θ dt+σ t dw; where x t f represents the image state at time t. θ The "denoising direction" of model learning is controlled by the policy parameter θ, σ t dw is a random noise term (i.e. a random perturbation term associated with Gaussian noise) used to introduce generative diversity.

[0042] In the flow matching image generation model, multiple initial noises are progressively denoised according to the stochastic differential equation at the first sampling time step (10 steps) to generate multiple candidate images; wherein, in each of the first sampling time steps, Gaussian noise perturbation is introduced to continuously inject randomness during the denoising process, balancing generation stability and diversity.

[0043] S130. In the stream matching image generation model, the advantage value corresponding to each candidate image is obtained by standard deviation normalization.

[0044] In this embodiment, in the stream matching image generation model, the strategy evaluation of each candidate image is performed by standard deviation normalization to obtain the advantage value corresponding to each candidate image.

[0045] In one embodiment, obtaining the advantage value corresponding to each candidate image through standard deviation normalization in the streaming matching image generation model includes: calculating the reward value corresponding to each candidate image in the streaming matching image generation model; and performing standard deviation normalization on the difference between the reward values ​​to obtain the advantage value corresponding to each candidate image.

[0046] In this embodiment, in the stream matching image generation model, each candidate image is traversed, and the reward value corresponding to each candidate image is calculated according to the reward function. The difference between each reward value is normalized by standard deviation to obtain the advantage value corresponding to each candidate image. This advantage value is used to measure the relative merits of different candidate images in the generation strategy when optimizing subsequent strategy parameters, and to assist the model in iterative update.

[0047] In one embodiment, calculating the reward value corresponding to each candidate image in the stream matching image generation model includes: calculating the reward value corresponding to each candidate image according to a reward function in the stream matching image generation model.

[0048] In this embodiment, in the stream matching image generation model, the reward value corresponding to each candidate image is calculated according to the reward function. The reward function can be combined with image quality evaluation indicators, such as sharpness and semantic fit with text prompts. If it is for a specific scenario, such as a medical scenario, medical professional feature compliance (such as lesion morphology standardization and image annotation consistency) will also be included.

[0049] S140. In the stream matching image generation model, the GRPO reinforcement learning mechanism is introduced to optimize the policy parameters based on the advantage value corresponding to each candidate image, so as to obtain the optimized policy parameters.

[0050] In this embodiment, a GRPO reinforcement learning mechanism is introduced into the stream matching image generation model to optimize the policy parameters based on the advantage value corresponding to each candidate image, resulting in optimized policy parameters. This invention significantly improves the accuracy of combined image generation and text rendering tasks by introducing the GRPO reinforcement learning mechanism. For example, in the GenEval benchmark test, the task accuracy is significantly improved from the original 63% to 95%, and the text rendering accuracy is improved from 59% to 92%, far exceeding the performance of similar models.

[0051] In one embodiment, step S140 includes: introducing KL regularization constraints, and optimizing the policy parameters using the advantage value corresponding to each candidate image according to the GRPO algorithm in the GRPO reinforcement learning mechanism to obtain optimized policy parameters.

[0052] In this embodiment, KL regularization constraints are introduced. Based on the GRPO algorithm in the GRPO reinforcement learning mechanism, the policy parameters are optimized using the advantage value corresponding to each candidate image to obtain the optimized policy parameters. The GRPO algorithm takes the form J... flow-GRPO =J CLIP -βD KL (π θnew ∥π θ ); where J flow-GRPO The objective function is used to optimize the policy parameter θ, with the optimization direction being to maximize this value, guiding the policy network to generate better images; J CLIP β is the reward function, used to measure the semantic matching degree between the generated image and the given prompt word; β is the policy stability coefficient (the range of values ​​is determined experimentally); D KL (π θnew ∥π θ The KL divergence between the optimized policy parameters θnew and the unoptimized policy parameters θ is used to quantify the difference between the two policy distributions. This invention introduces KL regularization constraints, which can prevent excessive policy parameter shifts and effectively suppress reward hijacking, ensuring that the final output image balances task score and visual quality.

[0053] S150. Based on the optimized strategy parameters, a high-quality image corresponding to the prompt word is generated through a stream matching image generation model.

[0054] In this embodiment, based on the optimized strategy parameter θnew, a high-quality image corresponding to the prompt word is generated through a stream matching image generation model.

[0055] In one embodiment, step S150 includes: converting the prompt word into initial Gaussian noise using a stream matching image generation model; in the stream matching image generation model, based on the SDE sampling strategy, and combined with the optimized strategy parameters, performing progressive denoising on multiple initial noises according to a preset second sampling time step to generate multiple candidate images.

[0056] In this embodiment, the prompt words are converted into initial Gaussian noise using a stream matching image generation model. In the stream matching image generation model, based on the SDE sampling strategy and combined with the optimized strategy parameters, multiple initial noises are gradually denoised according to a preset second sampling time step (40 steps) to generate multiple candidate images.

[0057] In summary, the embodiments of the present invention significantly improve the accuracy of combined image generation and text rendering tasks in complex tasks by leveraging the GRPO reinforcement learning mechanism and combining the randomness of the SDE sampling strategy with the optimization of strategy parameters. In terms of training efficiency, the use of a small number of first sampling time steps for gradual denoising shortens the training time without causing any loss to the final task performance.

[0058] Figure 2 This is a schematic block diagram of a stream matching image generation device based on online reinforcement learning optimization provided in an embodiment of the present invention. Figure 2 As shown, corresponding to the above-described stream matching image generation method based on online reinforcement learning optimization, this invention also provides a stream matching image generation apparatus based on online reinforcement learning optimization, the apparatus being configured in, as shown in... Figure 4 In this application environment, the user terminal communicates with the server via a network. For details, please refer to... Figure 2 The online reinforcement learning-optimized stream matching image generation device 700 includes:

[0059] The conversion unit 701 is used to convert the input prompt word into multiple initial noises using a stream matching image generation model;

[0060] The denoising unit 702 is used in the stream matching image generation model to gradually denoise multiple initial noises based on the SDE sampling strategy and in combination with predefined strategy parameters according to a preset first sampling time step, thereby generating multiple candidate images.

[0061] The acquisition unit 703 is used to acquire the advantage value corresponding to each candidate image by standard deviation normalization in the stream matching image generation model.

[0062] The optimization unit 704 is used to introduce the GRPO reinforcement learning mechanism in the stream matching image generation model, and optimize the policy parameters based on the advantage value corresponding to each candidate image to obtain the optimized policy parameters.

[0063] The generation unit 705 is used to generate a high-quality image corresponding to the prompt word based on the optimized strategy parameters and through a stream matching image generation model.

[0064] In some embodiments, when performing the step of converting the input prompt word into multiple initial noises using a stream matching image generation model, the conversion unit 701 is specifically used for:

[0065] In the flow matching image generation model, multiple initial noises are generated based on a standard Gaussian distribution for the input prompt words.

[0066] In some embodiments, when the denoising unit 702 performs the step of generating multiple candidate images in the streaming matching image generation model by progressively denoising multiple initial noises based on the SDE sampling strategy and in combination with predefined strategy parameters according to a preset first sampling time step, it is specifically used for:

[0067] In the stream matching image generation model, the SDE sampling strategy is combined with the strategy parameters to transform into a stochastic differential equation with strategy parameters; in the stream matching image generation model, multiple initial noises are progressively denoised according to the stochastic differential equation at the first sampling time step to generate multiple candidate images; wherein, in each of the first sampling time steps, Gaussian noise perturbation is introduced.

[0068] In some embodiments, when the acquisition unit 703 performs the step of obtaining the advantage value corresponding to each candidate image through standard deviation normalization in the streaming matching image generation model, it is specifically used for:

[0069] In the stream matching image generation model, a reward value is calculated for each candidate image; the difference between the reward values ​​is normalized by standard deviation to obtain the advantage value for each candidate image.

[0070] In some embodiments, when the acquisition unit 703 performs the step of calculating the reward value corresponding to each candidate image in the streaming matching image generation model, it is specifically used for:

[0071] In the stream matching image generation model, a reward value is calculated for each candidate image based on a reward function.

[0072] In some embodiments, when the optimization unit 704 performs the step of introducing the GRPO reinforcement learning mechanism in the streaming matching image generation model to optimize the policy parameters based on the advantage value corresponding to each candidate image, and obtains the optimized policy parameters, it is specifically used for:

[0073] By introducing KL regularization constraints and using the GRPO algorithm in the GRPO reinforcement learning mechanism, the policy parameters are optimized using the advantage value corresponding to each candidate image to obtain the optimized policy parameters.

[0074] In some embodiments, when the generation unit 705 performs the step of generating a high-quality image corresponding to the prompt word using a stream matching image generation model based on the optimized strategy parameters, it is specifically used for:

[0075] The prompt words are converted into initial Gaussian noise using a stream matching image generation model. In the stream matching image generation model, based on the SDE sampling strategy and combined with the optimized strategy parameters, multiple initial noises are gradually denoised according to a preset second sampling time step to generate multiple candidate images.

[0076] It should be noted that those skilled in the art can clearly understand that the specific implementation process of the above-mentioned online reinforcement learning-optimized stream matching image generation device and each unit can be referred to the corresponding description in the foregoing method embodiments. For the sake of convenience and brevity, it will not be repeated here.

[0077] The aforementioned stream matching image generation device based on online reinforcement learning optimization can be implemented as a computer program, which can, for example, Figure 3 It runs on the electronic device shown.

[0078] Please see Figure 3 , Figure 3 This is a schematic block diagram of an electronic device provided in an embodiment of the present invention. The electronic device 800 can be a terminal or a server. The terminal can be an electronic device with communication functions. The server can be a standalone server or a server cluster composed of multiple servers.

[0079] See Figure 3 The electronic device 800 includes a processor 802, a memory, and a network interface 805 connected via a system bus 801. The memory may include a non-volatile storage medium 803 and internal memory 804.

[0080] The non-volatile storage medium 803 may store an operating system 8031 ​​and a computer program 8032. The computer program 8032 includes program instructions that, when executed, cause the processor 802 to perform a stream matching image generation method based on online reinforcement learning optimization.

[0081] The processor 802 provides computing and control capabilities to support the operation of the entire electronic device 800.

[0082] The internal memory 804 provides an environment for the execution of the computer program 8032 in the non-volatile storage medium 803. When the computer program 8032 is executed by the processor 802, the processor 802 can execute a stream matching image generation method based on online reinforcement learning optimization.

[0083] This network interface 805 is used for network communication with other devices. Those skilled in the art will understand that... Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the electronic device 800 to which the present invention is applied. The specific electronic device 800 may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0084] The processor 802 is used to run a computer program 8032 stored in the memory to perform the following steps:

[0085] A streaming image generation model is used to transform the input prompt word into multiple initial noises. Within this model, based on the SDE sampling strategy and combined with predefined strategy parameters, the initial noises are progressively denoised at a preset first sampling time step to generate multiple candidate images. In the streaming image generation model, the advantage value corresponding to each candidate image is obtained through standard deviation normalization. Furthermore, a GRPO reinforcement learning mechanism is introduced to optimize the strategy parameters based on the advantage value corresponding to each candidate image, resulting in optimized strategy parameters. Based on these optimized strategy parameters, a high-quality image corresponding to the prompt word is generated using the streaming image generation model.

[0086] In some embodiments, when the processor 802 implements the step of converting the input prompt word into multiple initial noises using a stream matching image generation model, the specific steps are as follows:

[0087] In the flow matching image generation model, multiple initial noises are generated based on a standard Gaussian distribution for the input prompt words.

[0088] In some embodiments, when the processor 802 implements the step of generating multiple candidate images by progressively denoising multiple initial noises based on the SDE sampling strategy and in combination with predefined strategy parameters according to a preset first sampling time step in the stream matching image generation model, the specific implementation is as follows:

[0089] In the stream matching image generation model, the SDE sampling strategy is combined with the strategy parameters to transform into a stochastic differential equation with strategy parameters; in the stream matching image generation model, multiple initial noises are progressively denoised according to the stochastic differential equation at the first sampling time step to generate multiple candidate images; wherein, in each of the first sampling time steps, Gaussian noise perturbation is introduced.

[0090] In some embodiments, when implementing the step of obtaining the advantage value corresponding to each candidate image through standard deviation normalization in the streaming matching image generation model, the processor 802 specifically implements the following steps:

[0091] In the stream matching image generation model, a reward value is calculated for each candidate image; the difference between the reward values ​​is normalized by standard deviation to obtain the advantage value for each candidate image.

[0092] In some embodiments, when implementing the step of calculating the reward value corresponding to each candidate image in the streaming matching image generation model, the processor 802 specifically implements the following steps:

[0093] In the stream matching image generation model, a reward value is calculated for each candidate image based on a reward function.

[0094] In some embodiments, when the processor 802 implements the step of introducing the GRPO reinforcement learning mechanism in the streaming matching image generation model to optimize the policy parameters based on the advantage value corresponding to each candidate image, and obtains the optimized policy parameters, the specific implementation steps are as follows:

[0095] By introducing KL regularization constraints and using the GRPO algorithm in the GRPO reinforcement learning mechanism, the policy parameters are optimized using the advantage value corresponding to each candidate image to obtain the optimized policy parameters.

[0096] In some embodiments, when the processor 802 implements the step of generating a high-quality image corresponding to the prompt word based on the optimized strategy parameters and using a stream matching image generation model, the following steps are specifically implemented:

[0097] The prompt words are converted into initial Gaussian noise using a stream matching image generation model. In the stream matching image generation model, based on the SDE sampling strategy and combined with the optimized strategy parameters, multiple initial noises are gradually denoised according to a preset second sampling time step to generate multiple candidate images.

[0098] It should be understood that, in this embodiment of the invention, the processor 802 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0099] It will be understood by those skilled in the art that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program includes program instructions and can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiments of the above methods.

[0100] Therefore, the present invention also provides a storage medium. This storage medium can be a computer-readable storage medium. The storage medium stores a computer program, wherein the computer program includes program instructions. When executed by a processor, the program instructions cause the processor to perform the following steps:

[0101] A streaming image generation model is used to transform the input prompt word into multiple initial noises. Within this model, based on the SDE sampling strategy and combined with predefined strategy parameters, the initial noises are progressively denoised at a preset first sampling time step to generate multiple candidate images. In the streaming image generation model, the advantage value corresponding to each candidate image is obtained through standard deviation normalization. Furthermore, a GRPO reinforcement learning mechanism is introduced to optimize the strategy parameters based on the advantage value corresponding to each candidate image, resulting in optimized strategy parameters. Based on these optimized strategy parameters, a high-quality image corresponding to the prompt word is generated using the streaming image generation model.

[0102] In one embodiment, when the processor executes the program instructions to convert the input prompt word into multiple initial noise steps using a stream matching image generation model, it specifically implements the following steps:

[0103] In the flow matching image generation model, multiple initial noises are generated based on a standard Gaussian distribution for the input prompt words.

[0104] In one embodiment, when the processor executes the program instructions to implement the step of generating multiple candidate images in the stream matching image generation model by progressively denoising multiple initial noises based on the SDE sampling strategy and in combination with predefined strategy parameters according to a preset first sampling time step, the specific implementation is as follows:

[0105] In the stream matching image generation model, the SDE sampling strategy is combined with the strategy parameters to transform into a stochastic differential equation with strategy parameters; in the stream matching image generation model, multiple initial noises are progressively denoised according to the stochastic differential equation at the first sampling time step to generate multiple candidate images; wherein, in each of the first sampling time steps, Gaussian noise perturbation is introduced.

[0106] In one embodiment, when the processor executes the program instructions to implement the step of obtaining the advantage value corresponding to each candidate image through standard deviation normalization in the stream matching image generation model, it specifically implements the following steps:

[0107] In the stream matching image generation model, a reward value is calculated for each candidate image; the difference between the reward values ​​is normalized by standard deviation to obtain the advantage value for each candidate image.

[0108] In one embodiment, when the processor executes the program instructions to calculate the reward value corresponding to each candidate image in the stream matching image generation model, it specifically implements the following steps:

[0109] In the stream matching image generation model, a reward value is calculated for each candidate image based on a reward function.

[0110] In one embodiment, when the processor executes the program instructions to introduce the GRPO reinforcement learning mechanism in the stream matching image generation model and optimizes the policy parameters based on the advantage value corresponding to each candidate image to obtain the optimized policy parameters, the specific implementation steps are as follows:

[0111] By introducing KL regularization constraints and using the GRPO algorithm in the GRPO reinforcement learning mechanism, the policy parameters are optimized using the advantage value corresponding to each candidate image to obtain the optimized policy parameters.

[0112] In one embodiment, when the processor executes the program instructions to generate a high-quality image corresponding to the prompt word based on the optimized strategy parameters through a stream matching image generation model, the specific steps are as follows:

[0113] The prompt words are converted into initial Gaussian noise using a stream matching image generation model. In the stream matching image generation model, based on the SDE sampling strategy and combined with the optimized strategy parameters, multiple initial noises are gradually denoised according to a preset second sampling time step to generate multiple candidate images.

[0114] The storage medium can be any computer-readable storage medium capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), magnetic disk, or optical disk.

[0115] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0116] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of each unit is merely a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0117] The steps in the method of this invention can be adjusted, merged, or reduced in order according to actual needs. The units in the device of this invention can be merged, divided, or reduced according to actual needs. Furthermore, the functional units in the various embodiments of this invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0118] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0119] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for generating flow matching images based on online reinforcement learning optimization, characterized in that, The method includes: The input prompt words are converted into multiple initial noises using a stream matching image generation model; In the stream matching image generation model, based on the SDE sampling strategy, and combined with predefined strategy parameters, multiple initial noises are gradually denoised according to a preset first sampling time step to generate multiple candidate images. In the stream matching image generation model, the advantage value corresponding to each candidate image is obtained by standard deviation normalization; In the stream matching image generation model, the GRPO reinforcement learning mechanism is introduced to optimize the policy parameters based on the advantage value corresponding to each candidate image, so as to obtain the optimized policy parameters. Based on the optimized strategy parameters, a high-quality image corresponding to the prompt word is generated through a stream matching image generation model.

2. The stream matching image generation method based on online reinforcement learning optimization according to claim 1, characterized in that, The process of using a stream matching image generation model to convert the input prompt words into multiple initial noises includes: In the flow matching image generation model, multiple initial noises are generated based on a standard Gaussian distribution for the input prompt words.

3. The stream matching image generation method based on online reinforcement learning optimization according to claim 1, characterized in that, In the streaming matching image generation model, based on the SDE sampling strategy and combined with predefined strategy parameters, multiple initial noises are progressively denoised according to a preset first sampling time step to generate multiple candidate images, including: In the stream matching image generation model, the SDE sampling strategy is combined with the strategy parameters to transform it into a stochastic differential equation with strategy parameters; In the flow matching image generation model, multiple initial noises are progressively denoised according to the stochastic differential equation at the first sampling time step to generate multiple candidate images; wherein, in each of the first sampling time steps, Gaussian noise perturbation is introduced.

4. The stream matching image generation method based on online reinforcement learning optimization according to claim 1, characterized in that, In the streaming matching image generation model, the advantage value corresponding to each candidate image is obtained through standard deviation normalization, including: In the stream matching image generation model, a reward value is calculated for each candidate image; The difference between the reward values ​​is normalized by standard deviation to obtain the advantage value corresponding to each candidate image.

5. The stream matching image generation method based on online reinforcement learning optimization according to claim 4, characterized in that, In the stream matching image generation model, calculating the reward value corresponding to each candidate image includes: In the stream matching image generation model, a reward value is calculated for each candidate image based on a reward function.

6. The stream matching image generation method based on online reinforcement learning optimization according to claim 1, characterized in that, In the stream matching image generation model, a GRPO reinforcement learning mechanism is introduced to optimize the policy parameters based on the advantage value corresponding to each candidate image, resulting in optimized policy parameters, including: By introducing KL regularization constraints and using the GRPO algorithm in the GRPO reinforcement learning mechanism, the policy parameters are optimized using the advantage value corresponding to each candidate image to obtain the optimized policy parameters.

7. The stream matching image generation method based on online reinforcement learning optimization according to claim 1, characterized in that, The step of generating a high-quality image corresponding to the prompt word using a stream matching image generation model based on the optimized strategy parameters includes: The prompt words are converted into initial Gaussian noise using a stream matching image generation model; In the stream matching image generation model, based on the SDE sampling strategy and combined with the optimized strategy parameters, multiple initial noises are gradually denoised according to a preset second sampling time step to generate multiple candidate images.

8. A stream matching image generation device based on online reinforcement learning optimization, characterized in that, The device includes: The transformation unit is used to convert the input prompt words into multiple initial noises using a stream matching image generation model; The denoising unit is used in the stream matching image generation model to gradually denoise multiple initial noises based on the SDE sampling strategy and combined with predefined strategy parameters according to a preset first sampling time step, thereby generating multiple candidate images. The acquisition unit is used to acquire the advantage value corresponding to each candidate image by standard deviation normalization in the stream matching image generation model; The optimization unit is used to introduce the GRPO reinforcement learning mechanism in the stream matching image generation model, and optimize the policy parameters based on the advantage value corresponding to each candidate image to obtain the optimized policy parameters. The generation unit is used to generate a high-quality image corresponding to the prompt word based on the optimized strategy parameters and through a stream matching image generation model.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the stream matching image generation method based on online reinforcement learning optimization as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which includes program instructions that, when executed by a processor, cause the processor to perform the online reinforcement learning-optimized stream matching image generation method as described in any one of claims 1-7.

Images