Image processing device and method, model training device, and storage medium

Abstract

To provide an image processing device and method, a model training device, and a computer-readable storage medium.SOLUTION: An image processing device includes an image generation unit configured to set multiple motion-related parameters for input data so as to change the input data and generate a second image based on the input data in a change process using a pre-trained diffusion generative model. The multiple motion-related parameters include displacement, velocity, and acceleration. The pre-trained diffusion generative model is obtained by setting multiple motion-related parameters for pixel points so as to change each pixel point of each original training image in a training image set, and training a score-based diffusion generative model based on changed training images at a plurality of stages in the change process.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to the field of image processing, and in particular to an image processing apparatus and method, a model training apparatus, and a computer-readable storage medium. [Background technology]

[0002] With the development of technology, image-based techniques (e.g., image recognition) are widely used. Summary of the Invention [Problem to be solved by the invention]

[0003] The following presents a simplified summary of the disclosure in order to provide a basic understanding of aspects of the disclosure. It is not intended to be an exhaustive overview of the disclosure, nor is it intended to intentionally identify key or critical portions of the disclosure or to intentionally limit the scope of the disclosure. However, it is merely intended to introduce concepts in a simplified form as a prelude to the more detailed description that is presented later.

[0004] An object of the present disclosure is to provide an improved image processing apparatus and method, a model training apparatus, and a computer-readable storage medium. [Means for solving the problem]

[0005] One aspect of the present disclosure provides an image processing device including an image generation unit that sets a plurality of motion-related parameters for input data to change the input data, and generates a second image based on the input data in a change process using a pre-trained diffusion generative model, wherein the plurality of motion-related parameters include displacement, velocity, and acceleration, and the pre-trained diffusion generative model is obtained by setting the plurality of motion-related parameters for each pixel point in a training image set to change each pixel point of each original training image in the training image set, and training a score-based diffusion generative model based on changed training images at a plurality of stages in the change process.

[0006] Another aspect of the present disclosure provides a model training device, comprising: a processing unit for setting a plurality of motion-related parameters for each pixel point of each original training image in a training image set to change the pixel point; and training a score-based diffusion generative model based on the changed training images at a plurality of stages in the change process to obtain a trained diffusion generative model for generating images, wherein the plurality of motion-related parameters include displacement, velocity, and acceleration.

[0007] Another aspect of the present disclosure provides an image processing method including a step of setting a plurality of motion-related parameters for input data to vary the input data, and generating a second image based on the input data in a variation process using a pre-trained diffusion generative model, the plurality of motion-related parameters including displacement, velocity, and acceleration, and the pre-trained diffusion generative model is obtained by setting the plurality of motion-related parameters for each pixel point of each original training image in a training image set to vary the pixel point, and training a score-based diffusion generative model based on the varied training images at a plurality of stages in the variation process.

[0008] In another aspect of the present disclosure, there is further provided a computer program code and a computer program product for implementing the above-disclosed method, as well as a computer readable storage medium having recorded thereon a computer program code for implementing the above-disclosed method.

[0009] The following describes other aspects of the embodiments of the present disclosure, and in particular describes preferred embodiments of the present disclosure in detail, but the present disclosure is not limited to these embodiments. [Brief description of the drawings]

[0010] In order to make the principles and advantages of the present disclosure understandable, each embodiment of the present disclosure will be described with reference to the drawings. In all drawings, the same or similar components are indicated by the same or similar reference numerals. The drawings described herein are for illustrating preferred embodiments of the present disclosure and for interpreting the principles and advantages of the present disclosure, but are not all possible embodiments and do not limit the scope of the present disclosure. [Figure 1] FIG. 2 is a block diagram illustrating an example of a functional configuration of a model training device according to an embodiment of the present disclosure. [Diagram 2] FIG. 2 is a block diagram illustrating an example of a functional configuration of an image processing device according to an embodiment of the present disclosure. [Figure 3A] FIG. 2 illustrates a distribution of some stages when generating multiple 2D Swiss rolls using an image processing device 200 according to an embodiment of the present disclosure. [Figure 3B] FIG. 1 shows the distribution of some stages when generating multiple 2D Swiss rolls using a technique based on critically damped Langevin diffusion (CLD) (see, for example, Reference 1: T. Dockhorn, A. Vahdat, and K. Kreis. Score-based generative modeling with critically-damped langevin diffusion). [Figure 4A] 4A, 4B, and 4C are diagrams illustrating an example of a 2D Swiss roll generated using an image generating apparatus according to an embodiment of the present disclosure, a Variance Preserving SDE (VP-SDE) based technique (see, for example, Reference 2: Y. Song, J. Sohl-Dickstein, DP Kingma, A. Kumar, S. Ermon, and B. Poole. Score-based generative modeling through stochastic differential equations), and a CLD based technique, respectively. [Figure 4B]4A, 4B, and 4C are diagrams illustrating an example of a 2D Swiss roll generated using an image generating apparatus according to an embodiment of the present disclosure, a Variance Preserving SDE (VP-SDE) based technique (see, for example, Reference 2: Y. Song, J. Sohl-Dickstein, DP Kingma, A. Kumar, S. Ermon, and B. Poole. Score-based generative modeling through stochastic differential equations), and a CLD based technique, respectively. [Figure 4C] 4A, 4B, and 4C are diagrams illustrating an example of a 2D Swiss roll generated using an image generating apparatus according to an embodiment of the present disclosure, a Variance Preserving SDE (VP-SDE) based technique (see, for example, Reference 2: Y. Song, J. Sohl-Dickstein, DP Kingma, A. Kumar, S. Ermon, and B. Poole. Score-based generative modeling through stochastic differential equations), and a CLD based technique, respectively. [Diagram 5] FIG. 1 is a diagram showing a comparison of an image generating apparatus according to an embodiment of the present disclosure, a VP-SDE-based technique, and a CLD-based technique. [Figure 6] FIG. 2 is a block diagram illustrating an example of a functional configuration of an image processing device according to an embodiment of the present disclosure. [Figure 7] 1 is a flowchart illustrating an example of the flow of a model training method according to an embodiment of the present disclosure. [Figure 8] 1 is a flowchart showing an example of the flow of an image processing method according to an embodiment of the present disclosure. [Figure 9] 1 is a flowchart showing an example of the flow of an image processing method according to an embodiment of the present disclosure. [Figure 10] FIG. 1 is a block diagram showing an exemplary configuration of a personal computer applicable to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the drawings. For convenience of description, the specification does not show all the features of the actual embodiment. In addition, in order to realize the specific goal of the developer in the actual implementation, the specific embodiment may be modified, for example, the embodiment may be modified according to the constraints on the system and business. In addition, although the development work is very complicated and time-consuming, for those skilled in the art of this disclosure, this development work is merely an example work.

[0012] In order to clarify the present disclosure, the drawings show only the components or process steps of the apparatus closely related to the present disclosure, and omit details that are not relevant to the present disclosure.

[0013] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings.

[0014] <Example 1> 1 is a block diagram illustrating an example of a functional configuration of a model training apparatus 100 according to an embodiment of the present disclosure. As illustrated in FIG. 1, the model training apparatus 100 according to an embodiment of the present disclosure may include a processing unit 102.

[0015] The processing unit 102 may set a plurality of motion-related parameters for the pixel points to change each pixel point of each original training image in the training image set, and train a score-based diffusion generative model (e.g., may refer to Reference 2) based on the changed training images at a plurality of stages in the change process (which may be referred to as a "diffusion process") to obtain a trained score-based diffusion generative model (which may be referred to as a "trained diffusion generative model"). The trained diffusion generative model may be used to generate an image.

[0016] The above multiple motion-related parameters include displacement, velocity and acceleration. Compared with the technique of setting only displacement and / or velocity, the resulting trained diffusion generative model has better performance, e.g., the generated images have higher quality and better diversity.

[0017] The processing unit 102 may set noise for one or more of the above-mentioned multiple motion-related parameters in the change process as necessary. In some examples, the processing unit 102 may add random noise only to the acceleration of the multiple motion-related parameters. This can further improve the smoothness and speed of the change process.

[0018] For example, the process of changing the pixel point may be modeled by a third-order Langevin dynamics equation. For example, when random noise is added only to the acceleration among multiple motion-related parameters, the process of changing the pixel point may be expressed by the following equation (1).

[0019]

number

[0020] As can be clearly seen from equation (1), the random noise is tcan only directly affect the acceleration s t is the speed p t via position q t Indirectly affects q t The change in is relatively smooth. t In comparison, for the process described by Eq. (1), the position q t The smoothness of the change trajectory of is two orders of magnitude higher.

[0021] As will be understood by those skilled in the art, dq in Equation (1) t , dp t and / or ds t The noise-added motion-related parameters can be modified by adding or removing noise components to the equation corresponding to:

[0022] For example, U(q t )=L / 2||q t For example, the changed image at time t is (outside 4) TIFF2025031632000006.tif12170. Therefore, the above formula (1) can be expressed in matrix form as (outside 5) TIFF2025031632000007.tif12170 may be rewritten, where f(t) and G(t) may be expressed by the following equations (2) and (3), respectively.

[0023]

number

[0024]

number

[0025]

number

[0026]

number

[0027] When equation (1) is expressed by equations (2) and (3), the two Hamiltonian components (Outside 14) TIFF2025031632000020.tif12170 and (Outside 15) TIFF2025031632000021.tif12170 and one Ornstein-Uhlenbeck (OU) process (Outside 16) TIFF2025031632000022.tif13170 exists. This allows the diffusion process corresponding to equation (1) to proceed rapidly.

[0028] The key point of the training process is to obtain the score function. Based on Equation (4), the score function (Outside 17) TIFF2025031632000023.tif12170 may be expressed by the following equation (6).

[0029]

number

[0030] noise (outside 20) Since TIFF2025031632000027.tif8170 is always constrained within a small bounding box, noise prediction can further improve the performance of the obtained trained model compared with score prediction itself. Predicting noise in the training process can be equivalent to training a score-based diffusion generative model based on the difference between multiple stages of changed training images and the original training image.

[0031] for example, (outside 21) You can assume that the file is TIFF2025031632000028.tif8170. (outside 22) TIFF2025031632000029.tif11170, (outside 23) TIFF2025031632000030.tif8170, where α<<1. Therefore, (outside 24) TIFF2025031632000031.tif8170. In this case, the score function based on Eq. (6) (Outside 25) TIFF2025031632000032.tif10170 may be expressed by the following equation (7).

[0032]

number

[0033]

number

[0034]

number

[0035]

number

[0036]

number

[0037] In some embodiments, in addition to displacement, velocity and acceleration, other motion-related parameters may be included, which can further improve the performance of the resulting trained diffusion model.

[0038] For example, in this case, the process of changing the pixel points may be expressed by the following equation (12).

[0039]

number

[0040] <Example 2> 2 is a block diagram showing an example of a functional configuration of an image processing device 200 according to an embodiment of the present disclosure. As shown in FIG. 2, the image processing device 200 according to the embodiment of the present disclosure may include an image generating unit 202.

[0041] The image generator 202 may set a plurality of motion-related parameters for the input data so as to change the input data, and generate a second image based on the input data in the change process using a pre-trained diffusion generation model. The above-mentioned plurality of motion-related parameters may include displacement, velocity, and acceleration. For example, the input data may be noise, a specific data distribution, etc.

[0042] For example, the pre-trained diffusion generative model is obtained by setting a plurality of motion-related parameters for each pixel point to change each pixel point of each original training image in the training image set, and training a score-based diffusion generative model based on the changed training images at a plurality of stages in the change process. For example, the pre-trained diffusion generative model may be a trained diffusion generative model trained by the above-mentioned model training device 100.

[0043] By setting multiple motion-related parameters including displacement, velocity and acceleration, the quality and variety of the obtained second images may be higher, for example, as compared to techniques that set only displacement and / or velocity.

[0044] The image generating unit 202 may set noise for one or more of the above-mentioned multiple motion-related parameters as necessary during the change process. In some examples, the image generating unit 202 may add random noise only to the acceleration of the multiple motion-related parameters. This can further improve the smoothness and speed of the change process, and further improve the quality of the second image, etc.

[0045] For example, the operation of setting a plurality of motion-related parameters for the input data to change the input data may be expressed as follows:

[0046]

number

[0047] p(x 0 , 0) is the initial distribution (i.e., the input data with the initial velocity and acceleration set). The distribution p(x t ,t) (i.e., data during the change process) is (Outside 32) TIFF2025031632000046.tif11170, where: (Outside 33) TIFF2025031632000047.tif13170 is the Fokker-Planck operator.

[0048] The above equation (13) may be rewritten as the following equation (14).

[0049]

number

[0050]

number

[0051]

number

[0052] Where: (outside 40) When solving TIFF2025031632000057.tif12170, it is necessary to calculate the mean and variance of the data after time t / 2P. (outside 41) TIFF2025031632000058.tif16170 is (outside 42) The average value u obtained by integrating the following equations (17) and (18) in TIFF2025031632000059.tif14170 t and variance Σ t is a Gaussian distribution with

[0053]

number

[0054]

number

[0055]

number

[0056]

number

[0057]

number

[0058] 4A, 4B, and 4C show an example of a 2D Swiss roll produced by the image processing device 200 according to an embodiment of the present disclosure, a Variance Preserving SDE (VP-SDE) based technique, and a CLD based technique, respectively. As can be seen by comparing Figs. 4A, 4B, and 4C, the quality of the Swiss roll produced by the image processing device 200 according to an embodiment of the present disclosure is higher than that of the VP-SDE based technique and the CLD based technique.

[0059] FIG. 5 is a diagram showing a comparison between the image processing device 200 according to an embodiment of the present disclosure, the VP-SDE-based technique, and the CLD-based technique. In FIG. 5, n represents function evaluation. As can be seen from FIG. 5, the FID (Frechet Inception Distance) corresponding to the image processing device 200 is lower than both the VP-SDE-based technique and the CLD-based technique. That is, it shows that the image generated using the image processing device 200 is closer to the real image than the VP-SDE-based technique and the CLD-based technique.

[0060] Furthermore, according to the inventors' experimental findings, the image processing device 200 according to an embodiment of the present invention is less affected by the hyperparameters L and α (initial velocity and acceleration variable scales), and can therefore be easily migrated to other generation tasks, such as 3D data generation tasks.

[0061] In some examples, the image generating unit 202 may refer to the first image and generate a second image based on the input data in the change process. The first image may be set according to the needs of the user. As an example, when the image generating unit 202 is used to generate an image of a specific type of object (e.g., an object, an animal, etc.), the first image may be an image corresponding to the type of object.

[0062] As another example, the image generator 202 may be used to generate a second version of an image (an example of a second image) based on a first version of an image (an example of a first image). For example, the first version of an image may be a lower resolution image and the second version of an image may be a higher resolution image. For example, the first version of an image may be a blurred or occluded image and the second version of an image may be a clearer or less occluded (or no occlusion) image.

[0063] For example, when the image generating unit 202 generates a second image by referring to a first image, the above formula (13) may be rewritten as the following formulas (22) to (24).

[0064]

number

[0065]

number

[0066]

number

[0067] It should be noted that although the above describes some examples of the formulas of the present disclosure, in practical applications, those skilled in the art may adopt other formulas based on the principles of the present disclosure.

[0068] <Example 3> 6 is a block diagram showing an example of a functional configuration of an image processing device 600 according to an embodiment of the present disclosure. As shown in FIG. 6, the image processing device 600 according to an embodiment of the present disclosure may include an image generation unit 602 and an image recognition unit 604.

[0069] The image generating unit 602 may set a plurality of motion-related parameters for the input data so as to change the input data, and generate a second image based on the input data in the change process using a pre-trained diffusion generation model. The above-mentioned plurality of motion-related parameters may include displacement, velocity, and acceleration. For example, the image generating unit 602 may have the same function and configuration as the image generating unit 202, for example, and therefore will be briefly described below.

[0070] The image recognition unit 604 may recognize objects (eg, faces, objects, animals, etc.) associated with the second image generated by the image generation unit 602.

[0071] For example, the image processing device 600 may be realized as an electronic device (e.g., a mobile phone, an automatic register, etc.) or may be incorporated into an electronic device. For example, the image generation unit 602 may generate a second image by referring to a first image acquired by an electronic device (e.g., an image acquisition device incorporated into an electronic device). For example, when the first image acquired by the electronic device is blurred or obscured, the image generation unit 602 may generate a second image that is clear or has less obscuration (or no obscuration) corresponding to the first image, and provide the generated second image to the image recognition unit 604. This can improve the accuracy of image recognition.

[0072] For example, the image processing device 600 may be realized as a cash register or may be incorporated in a cash register. For example, the cash register may further include an image capture device for capturing an image of an item. The image generation unit 602 processes the image of the item captured by the image capture device to generate a second image. The image recognition unit 604 may recognize the second image to obtain a category of the item for the cash register to determine the price of the item. This allows the category of the item to be accurately recognized even when the item is occluded or the image of the item is not clear.

[0073] For example, the image processing device 600 may be realized as or incorporated into an identification device. For example, the identification device may include an image capture device for capturing an image of a person or a part of the person (e.g., a face). The image generation unit 602 may process the image captured by the image capture device to generate a second image. The image recognition unit 604 may recognize the second image to capture the identity of the person. This allows the identity of the person to be accurately recognized even if the captured image is not clear.

[0074] <Example 4> 7 is a flowchart showing an example of a flow of a model training method according to an embodiment of the present disclosure. As shown in FIG 7, a model training method 700 according to an embodiment of the present disclosure may start in step S702 and end in step S706.

[0075] As shown in FIG. 7, the model training method 700 may include a process step S704. In the process step S704, a plurality of motion-related parameters may be set for a pixel point to change each pixel point of each original training image in a training image set, and a score-based diffusion generation model may be trained based on the changed training images at a plurality of stages in the change process to obtain a trained diffusion generation model. The trained diffusion generation model may be used to generate an image. For example, the process step S704 may be performed by the above-mentioned processing unit 102, and will be briefly described below.

[0076] The above multiple motion-related parameters include displacement, velocity and acceleration. Compared with the technique of setting only displacement and / or velocity, the resulting trained diffusion generative model has better performance, e.g., the generated images have higher quality and better diversity.

[0077] In some examples, random noise may be added only to the acceleration among the above multiple motion-related parameters, which can further improve the smoothness and speed of the change process.

[0078] <Example 5> 8 is a flowchart showing an example of the flow of an image processing method according to an embodiment of the present disclosure. As shown in FIG 8, an image processing method 800 according to an embodiment of the present disclosure may start in step S802 and end in step S806.

[0079] As shown in Fig. 8, the image processing method 800 may include an image generating step S804. In the image generating step S804, a plurality of motion-related parameters may be set for the input data so as to change the input data, and a second image may be generated based on the input data in the change process using a pre-trained diffusion generation model. The above-mentioned plurality of motion-related parameters may include displacement, velocity, and acceleration. For example, the image generating step S804 may be performed by the above-mentioned image generating unit 202, and therefore will be briefly described below.

[0080] For example, the pre-trained diffusion generative model is obtained by setting a plurality of motion-related parameters for each pixel point to change each pixel point of each original training image in the training image set, and training a score-based diffusion generative model based on the changed training images at a plurality of stages in the change process. For example, the pre-trained diffusion generative model may be a trained diffusion generative model trained by the above-mentioned model training device 100.

[0081] By setting multiple motion-related parameters including displacement, velocity and acceleration, the quality and variety of the obtained second images may be higher, for example, as compared to techniques that set only displacement and / or velocity.

[0082] <Example 6> 9 is a flowchart showing an example of the flow of an image processing method according to an embodiment of the present disclosure. As shown in FIG 9, an image processing method 900 according to an embodiment of the present disclosure may start in step S902 and end in step S908.

[0083] As shown in FIG. 9, an image processing method 900 according to an embodiment of the present disclosure may include an image generating step S904 and an image recognizing step S906.

[0084] In the image generating step S904, a plurality of motion-related parameters may be set for the input data so as to change the input data, and a second image may be generated based on the input data in the change process using a pre-trained diffusion generation model. The above-mentioned plurality of motion-related parameters may include displacement, velocity, and acceleration. For example, the image generating step S904 may be performed by the above-mentioned image generating unit 602, and therefore, the description thereof will be omitted below.

[0085] In the image recognition step S906, an object (e.g., a face, an object, an animal, etc.) related to the second image generated in the image generation step S904 may be recognized. For example, the image recognition step S906 may be performed by the above-mentioned image recognition unit 604, and therefore, a description thereof will be omitted below.

[0086] Although the functional configuration and operation of the information processing device and the information processing method according to the embodiments of the present disclosure have been described above, the functional configuration and operation are merely illustrative and do not limit the present disclosure. Those skilled in the art may modify the above embodiments according to the principles of the present disclosure, for example, may add, delete, or combine functional modules in each embodiment, and these modifications are within the scope of the present disclosure.

[0087] Furthermore, since the device embodiments herein correspond to the method embodiments described above, for contents that are not described in detail in the device embodiments, reference may be made to the corresponding explanations of the method embodiments described above, and such explanations will be omitted here.

[0088] In addition, the present disclosure further provides a storage medium and a program product. The device-executable instructions in the storage medium and the program product according to the embodiments of the present disclosure may execute the above-mentioned image processing method and model training method, and the contents not described in detail herein may refer to the corresponding descriptions in the embodiments of the above-mentioned methods, and the description thereof will be omitted here.

[0089] Accordingly, the present disclosure further includes a storage medium having a program product recorded thereon, the program product including machine-executable instructions, including, but not limited to, a floppy disk, an optical disk, a magneto-optical disk, a memory card, a memory stick, and the like.

[0090] The above-mentioned processes and devices may be realized by software and / or firmware. When implemented by software and / or firmware, a program constituting software for implementing the above-mentioned method may be installed from a storage medium or a network into a computer having a dedicated hardware configuration, for example, a general-purpose personal computer 1000 shown in Fig. 10, and the computer can execute various functions when various programs are installed.

[0091] 10, a central processing unit (CPU) 1001 executes various processes according to programs stored in a read-only memory (ROM) 1002 or programs loaded from a storage unit 1008 to a random access memory (RAM) 1003. The RAM 1003 stores data necessary for the CPU 1001 to execute various processes as necessary.

[0092] The CPU 1001, the ROM 1002, and the RAM 1003 are connected to one another via a bus 1004. An input / output interface 1005 is also connected to the bus 1004.

[0093] An input unit 1006 (including a keyboard, a mouse, etc.), an output unit 1007 (including a display, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.), a storage unit 1008 (including, for example, a hard disk, etc.), and a communication unit 1009 (including, for example, a network interface card, for example, a LAN card, a modem, etc.) are connected to the input / output interface 1005. The communication unit 1009 executes communication processing via a network, for example, the Internet.

[0094] If necessary, the driver 1010 may be connected to the input / output interface 1005. The removable medium 1011 is, for example, a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like, and is set up in the driver 1010 as necessary, and a computer program read from the removable medium 1011 is installed in the storage unit 1008 as necessary.

[0095] When the above-mentioned processes are carried out by software, a program constituting the software is installed via a network, such as the Internet, or a storage medium, such as a removable medium 1011 .

[0096] It should be noted that these storage media are not limited to the removable medium 1011 shown in Fig. 10, which stores the program and provides the program to the user separately from the device. The removable medium 1011 includes, for example, a magnetic disk (including a floppy disk (registered trademark)), an optical disk (including an optical disk-read only memory (CD-ROM) and a digital versatile disk (DVD)), a magneto-optical disk (a mini disk (MD) (registered trademark)), and a semiconductor memory. Alternatively, the storage medium may be a ROM 1002, a hard disk included in the storage unit 1008, or the like, which stores the program and is provided to the user together with the device including them.

[0097] The above describes the preferred embodiments of the present disclosure with reference to the drawings, but the above embodiments and examples are illustrative and not restrictive. Those skilled in the art may make various modifications, improvements, and equivalent changes to the present disclosure within the spirit and scope of the claims. These modifications, improvements, and equivalent changes are included in the protection scope of the present disclosure.

[0098] For example, the functions included in one unit in the above embodiments may be realized by separate devices. Also, the functions realized by the units in the above embodiments may each be realized by a separate device. Furthermore, one of the above functions may be realized by multiple units. These configurations are within the scope of the present disclosure.

[0099] In addition, the methods of the present disclosure are not limited to being performed in the chronological order described in the specification, and may be performed sequentially, in parallel, or independently in other chronological orders, and therefore the order of execution of the methods described herein does not limit the technical scope of the present disclosure.

[0100] Furthermore, the following supplementary notes are disclosed regarding the embodiments including the above-mentioned examples, but the present invention is not limited to these supplementary notes. (Appendix 1) an image generating unit that sets a plurality of motion-related parameters for the input data so as to change the input data, and generates a second image based on the input data in the process of change using a pre-trained diffusion generative model; the plurality of motion-related parameters include displacement, velocity, and acceleration; The image processing apparatus, wherein the pre-trained diffusion generative model is obtained by setting the plurality of motion-related parameters for each pixel point of each original training image in a training image set so as to change the pixel point, and training a score-based diffusion generative model based on the changed training images at a plurality of stages in the change process. (Appendix 2) 2. The image processing device according to claim 1, wherein the image generating unit adds random noise only to the acceleration during the change process of the plurality of motion-related parameters. (Appendix 3) 2. The image processing device of claim 1, wherein generating a second image includes generating the second image based on input data in a changing process by referring to the first image. (Appendix 4) 4. The image processing device according to claim 1, further comprising an image recognition unit that recognizes an object related to the second image. (Appendix 5) a processor for setting a plurality of motion-related parameters for each pixel point of each original training image in a training image set to change the pixel point, and training a score-based diffusion generative model based on the changed training images at a plurality of stages in the change process to obtain a trained diffusion generative model for generating an image; The plurality of motion-related parameters include displacement, velocity and acceleration. (Appendix 6) The model training apparatus of claim 5, wherein the processing unit trains a diffusion generative model based on the score based on a difference between the training images with the multiple stages of change and an original training image. (Appendix 7) 7. The model training device according to claim 5, wherein the processing unit adds random noise only to the acceleration during the change process of the plurality of movement-related parameters. (Appendix 8) setting a plurality of motion-related parameters for the input data to vary the input data, and generating a second image based on the input data in the variation process using a pre-trained diffusion generative model; the plurality of motion-related parameters include displacement, velocity, and acceleration; The image processing method, wherein the pre-trained diffusion generative model is obtained by setting the plurality of motion-related parameters for each pixel point of each original training image in a training image set to change the pixel point, and training a score-based diffusion generative model based on the changed training images at multiple stages in the change process. (Appendix 9) 9. The image processing method of claim 8, further comprising the step of adding random noise to the acceleration during the change. (Appendix 10) The image processing method according to claim 8 or 9, wherein generating a second image includes generating the second image based on input data in a changing process by referring to the first image. (Appendix 11) 10. The image processing method of claim 8 or 9, further comprising a step of recognizing an object associated with the second image. (Appendix 12) A computer readable storage medium having instructions stored thereon which, when executed by a computer, cause the computer to perform an image processing method according to any one of claims 8 to 11.

Claims

1. an image generating unit that sets a plurality of motion-related parameters for the input data so as to change the input data, and generates a second image based on the input data in the process of change using a pre-trained diffusion generative model; the plurality of motion-related parameters include displacement, velocity, and acceleration; The image processing apparatus, wherein the pre-trained diffusion generative model is obtained by setting the plurality of motion-related parameters for each pixel point of each original training image in a training image set so as to change the pixel point, and training a score-based diffusion generative model based on the changed training images at a plurality of stages in the change process.

2. The image processing device according to claim 1 , wherein the image generating section adds random noise only to the acceleration during the change process for the plurality of movement-related parameters.

3. The image processing device according to claim 1 , wherein generating the second image includes generating the second image based on input data in a changing process with reference to the first image.

4. The image processing device according to claim 1 , further comprising an image recognition unit that recognizes an object related to the second image.

5. a processor for setting a plurality of motion-related parameters for each pixel point of each original training image in a training image set to change the pixel point, and training a score-based diffusion generative model based on the changed training images at a plurality of stages in the change process to obtain a trained diffusion generative model for generating an image; The plurality of motion-related parameters include displacement, velocity and acceleration.

6. The model training apparatus according to claim 5 , wherein the processing unit trains the score-based diffusion generative model based on differences between the training images with the multiple stages of change and original training images.

7. The model training device according to claim 5 , wherein the processing unit adds random noise only to the acceleration during the change process of the plurality of movement-related parameters.

8. setting a plurality of motion-related parameters for the input data to vary the input data, and generating a second image based on the input data in the variation process using a pre-trained diffusion generative model; the plurality of motion-related parameters include displacement, velocity, and acceleration; The image processing method, wherein the pre-trained diffusion generative model is obtained by setting the plurality of motion-related parameters for each pixel point of each original training image in a training image set to change the pixel point, and training a score-based diffusion generative model based on the changed training images at multiple stages in the change process.

9. 9. The image processing method according to claim 8, further comprising the step of adding random noise to the acceleration during the change.

10. A computer readable storage medium having instructions stored thereon which, when executed by a computer, cause the computer to perform the image processing method of claim 8 or 9.

Images