Image display device

Abstract

The present invention provides an image display device comprising: an acquisition unit that acquires a plurality of images that correspond in a 1:1 manner to a plurality of processes based on a prompt, the images having been determined as a result of executing the plurality of processes in a time series; a display control unit that causes a display device to display a user interface image for accepting a user operation; a detection unit that detects the user operation with respect to the user interface image; a selection unit that selects, on the basis of the user operation detected by the detection unit, a first image from among the plurality of images that correspond in a 1:1 manner to the plurality of processes; and an identification unit that identifies, from among the plurality of processes, a process whose corresponding image fulfills a first criterion, wherein the display control unit causes the display device to display the first image, and the user interface image includes a mark that corresponds to the process identified by the identification unit.

Description

Image display device

[0001] The present invention relates to an image display device.

[0002] 2. Description of the Related Art When an image display device selects and displays one or more images from a plurality of images, a slide bar may be used as an example of a user interface displayed on the display device.

[0003] For example, the information processing device of Patent Document 1 switches the display of images between a plurality of images that mainly include first images and a plurality of images that include more second images than first images, in response to a slide operation on a slide bar.

[0004] Japanese Patent Application Laid-Open No. 2022-21316

[0005] However, in the technology disclosed in Patent Document 1, when switching between multiple images to be displayed, the user of the information processing device is unable to intuitively grasp the change in image quality among the multiple images.

[0006] An object of the present disclosure is to provide an image display device that allows a user to intuitively grasp changes in image quality among a plurality of images when switching between the images to be displayed.

[0007] The image display device of the present disclosure includes an acquisition unit that acquires an image corresponding to each of a plurality of processes determined by executing a plurality of processes based on a prompt in chronological order, a display control unit that causes the display device to display a user interface image that accepts user operations, a detection unit that detects the user's operation on the user interface image, a selection unit that selects a first image from among the images corresponding to the plurality of processes based on the user operation detected by the detection unit, and an identification unit that identifies a process from the plurality of processes whose corresponding image quality satisfies a first standard, wherein the display control unit causes the display device to display the first image, and the user interface image includes a mark corresponding to the process identified by the identification unit.

[0008] According to the present disclosure, when switching between a plurality of images to be displayed, it is possible to intuitively grasp the change in image quality among the plurality of images.

[0009] FIG. 1 is a diagram showing the overall configuration of an image display system 1. FIG. 2 is a block diagram showing an example of the configuration of an image generation server 20. FIG. 3 is a block diagram showing an example of an image generation model IGM. FIG. 4 is a block diagram showing an example of the configuration of an image display device 10[k]. FIG. 5 is a diagram showing an example of a user interface image UI. FIG. 6 is a diagram showing an example of a user interface image UI. FIG. 7 is a diagram showing an example of a user interface image UIA. FIG. 8 is a diagram showing an example of a user interface image UIA. FIG. 9 is a diagram showing an example of a user interface image UIA. FIG. 10 is a sequence diagram showing the operation of the image display system 1. FIG. 11 is a block diagram showing an example of an image generation model IGMA. FIG. 12 is a diagram showing an example of a user interface image UIA. FIG. 13 is a diagram showing an example of a user interface image UIA. FIG. 14 is a diagram showing an example of a user interface image UIA.

[0010] As described above, with conventional image display technologies, when switching between multiple images to be displayed, it is not possible to intuitively grasp changes in image quality among the multiple images. An image display device according to the present disclosure identifies processes that result in image quality that meet a first criterion. A user interface image displayed by the image display device includes a mark corresponding to the identified process. This allows a user of the image display device according to the present disclosure to intuitively grasp changes in image quality.

[0011] 1: First Embodiment An image display system 1 according to a first embodiment will be described below with reference to Figures 1 to 8. Note that in each figure, the dimensions and scale of each part are appropriately different from the actual ones. Furthermore, the embodiments described below are preferred specific examples, and therefore various technically preferable limitations are applied, but the scope of the present disclosure is not limited to these forms unless otherwise specified in the following description to the effect that the present disclosure is limited.

[0012] 1-1: Configuration of First Embodiment 1-1-1: Overall Configuration FIG. 1 is a diagram showing the overall configuration of an image display system 1 according to this embodiment. The image display system 1 includes n image display devices 10[1] to 10[n] and one image generation server 20. n is an integer equal to or greater than 1. The n image display devices 10[1] to 10[n] and the one image generation server 20 are communicably connected to each other via a communication network NET. In FIG. 1, a user U[k] uses the image display device 10[k]. k is an integer equal to or greater than 1 and equal to or less than n.

[0013] The image generation server 20 is a server that generates images to be displayed on the image display devices 10[1] to 10[n]. The image generation server 20 collects various information related to the image generation service from each of the image display devices 10[1] to 10[n] via the communication network NET. The image generation server 20 also generates images based on the various information. Furthermore, the image generation server 20 transmits the generated images to each of the image display devices 10[1] to 10[n].

[0014] The image display devices 10[1] to 10[n] display images received from the image generation server 20. The image display devices 10[1] to 10[n] include personal computers, tablet terminals, smartphones, smartwatches, etc. Note that a smartwatch may be connected to the communication network NET via an image display device 10 other than a smartwatch.

[0015] That is, the image display system 1 is a system that provides an image display service to any user U[k] who uses the image display devices 10[1] to 10[n].

[0016] 1-1-2: Configuration of Image Generation Server Fig. 2 is a block diagram showing an example configuration of the image generation server 20 in Fig. 1. As shown in Fig. 2, the image generation server 20 includes a processing device 21, a storage device 22, a display 23, an input device 24, and a communication device 25. The elements included in the image generation server 20 are connected to each other by one or more buses for communicating information.

[0017] The processing device 21 is a processor that controls the entire image generation server 20. The processing device 21 is configured, for example, using one or more chips. The processing device 21 is configured, for example, using a central processing unit (CPU) that includes an interface with peripheral devices, an arithmetic unit, and a register. Note that some or all of the functions of the processing device 21 may be realized by hardware such as a digital signal processor (DSP), an application specific integrated circuit (ASIC), a programmable logic device (PLD), and a field programmable gate array (FPGA). The processing device 21 executes various processes in parallel or sequentially.

[0018] The storage device 22 is a recording medium that can be read and written by the processing device 21. The storage device 22 includes, for example, a nonvolatile memory and a volatile memory. The nonvolatile memory is, for example, a ROM, an EPROM, and an EEPROM. The volatile memory is, for example, a RAM.

[0019] The storage device 22 stores a plurality of programs, including a control program PR2, and an image generation model IGM, to be executed by the processing device 21. The storage device 22 also functions as a work area for the processing device 21.

[0020] The display 23 is a device that displays images and text information. The display 23 displays various images under the control of the processing device 21. For example, various display panels such as a liquid crystal panel and an organic EL panel are suitably used as the display 23.

[0021] The input device 24 is a device that accepts operations by an administrator of the image display system 1. For example, the input device 24 includes a keyboard, a touchpad, a touch panel, or a pointing device such as a mouse. Here, if the input device 24 includes a touch panel, it may also serve as the display 23.

[0022] The communication device 25 is hardware serving as a transmitting / receiving device for communicating with other devices. The communication device 25 is also referred to as, for example, a network device, a network controller, a network card, or a communication module. The communication device 25 may include a connector for wired connection and an interface circuit corresponding to the connector. The communication device 25 may also include a wireless communication interface. Examples of the connector and interface circuit for wired connection include products that comply with wired LAN, IEEE 1394, and USB. Examples of the wireless communication interface include products that comply with wireless LAN, Bluetooth (registered trademark), etc.

[0023] The processing device 21 functions as a communication control unit 211 and a generation unit 212 by, for example, reading and executing a control program PR2 from the storage device 22.

[0024] The communication control unit 211 causes the communication device 25 to transmit and receive various types of information to and from the image display device 10. The various types of information include, for example, a prompt that the user U[k] inputs to the image display device 10[k] in order to have the image generation server 20 generate an image. A "prompt" is a keyword that the image generation server 20 inputs to the image generation model IGM, which will be described later, in order to generate an image GI. The communication control unit 211 causes the communication device 25 to receive the prompt from the image display device 10[k]. The prompt is, for example, the following prompt: "Little bird, beak, holding in mouth"

[0025] The generation unit 212 acquires the prompt received by the communication device 25. The generation unit 212 then executes processing based on the prompt using an image generation model IGM to generate an image corresponding to the processing. Image generation models IGM are well known, and examples of such models include DALL E 2 (https: / / openai.com / dall-e-2) and Stable Diffusion (https: / / stablediffusionweb.com / ).

[0026] In this embodiment, the image generation model IGM executes a process having multiple steps in chronological order, outputting one image corresponding to each step. For example, if the image generation model IGM is a stable diffusion model, it outputs a new image by removing noise from the original image at each step.

[0027] 3 is a functional block diagram showing an example of the functions of the generation unit 212. The generation unit 212 reads out the image generation model IGM from the storage device 22 and executes it, thereby functioning as a prompt acquisition unit 501, an original image acquisition unit 502, a correct image prediction unit 503, a predicted noise generation unit 504, a noise removal unit 505, and a step number acquisition unit 506.

[0028] The prompt acquisition unit 501 acquires a prompt received by the communication device 25 .

[0029] The original image acquisition unit 502 acquires an original image from the storage device 22. The original image is an image consisting of only noise.

[0030] The correct image prediction unit 503 predicts a correct image that is an image that should be generated from the prompt.

[0031] The predicted noise generating unit 504 generates predicted noise, which is noise that should be removed in order to convert the original image into a correct image.

[0032] The noise removal unit 505 removes the predicted noise from the original image to generate a new image. The original image acquisition unit 502 then acquires the new image from which the predicted noise has been removed by the noise removal unit 505 as a new original image.

[0033] The step number acquisition unit 506 acquires the number of steps to be executed in chronological order, with a set of a series of operations from the original image acquisition unit 502 to the noise removal unit 505 being considered as one step. For example, the user U[k] inputs the number of steps to the image display device 10[k] using the input device 14. The step number acquisition unit 506 acquires the number of steps input by the user U[k] from the image display device 10[k]. Alternatively, the step number acquisition unit 506 may acquire a predetermined number of steps that is pre-stored in the storage device 22.

[0034] The generation unit 212 treats the series of operations from the original image acquisition unit 502 to the noise removal unit 505 as one process, and generates one image corresponding to each process by executing the processing operations in chronological order the number of times acquired by the step number acquisition unit 506.

[0035] The communication control unit 211 causes the image display device 10[k] to transmit, to the communication device 25, a plurality of images that are generated by the generation unit 212 and correspond one-to-one to a plurality of processes.

[0036] 1-1-3: Configuration of Image Display Device FIG. 4 is a block diagram showing an example configuration of the image display device 10[k] in FIG. 1. Note that in FIG. 1, the configuration of the image display device 10[k] and the configuration of the other image display devices 10 may be the same as or different from each other. As shown in FIG. 4, the image display device 10 includes a processing device 11, a storage device 12, a display 13, an input device 14, and a communication device 15. The elements included in the image display device 10 are connected to each other by a single or multiple buses for communicating information.

[0037] The processing device 11 is a processor that controls the entire image display device 10. The processing device 11 is configured, for example, using one or more chips. The processing device 11 is configured, for example, using a central processing unit (CPU) that includes an interface with peripheral devices, an arithmetic unit, and a register. Note that some or all of the functions of the processing device 11 may be realized by hardware such as a DSP, an ASIC, a PLD, and an FPGA. The processing device 21 executes various processes in parallel or sequentially.

[0038] The storage device 12 is a recording medium that can be read and written by the processing device 11. The storage device 12 includes, for example, a nonvolatile memory and a volatile memory. The nonvolatile memory is, for example, a ROM, an EPROM, and an EEPROM. The volatile memory is, for example, a RAM.

[0039] The storage device 12 stores a plurality of programs including a control program PR1 to be executed by the processing device 11. The storage device 12 also functions as a work area for the processing device 11.

[0040] The display 13 is a device that displays images and text information. The display 13 displays various images under the control of the processing device 11. For example, various display panels such as a liquid crystal panel and an organic EL panel are suitably used as the display 13. The display 13 is an example of a "display device."

[0041] The input device 14 is a device that accepts operations by an administrator of the image display system 1. For example, the input device 14 includes a keyboard, a touchpad, a touch panel, or a pointing device such as a mouse. Here, if the input device 14 includes a touch panel, it may also serve as the display 13.

[0042] The communication device 15 is hardware serving as a transmitting / receiving device for communicating with other devices. The communication device 15 is also referred to as, for example, a network device, a network controller, a network card, or a communication module. The communication device 15 may include a connector for wired connection and an interface circuit corresponding to the connector. The communication device 15 may also include a wireless communication interface. Examples of the connector and interface circuit for wired connection include products that comply with wired LAN, IEEE 1394, and USB. Examples of the wireless communication interface include products that comply with wireless LAN, Bluetooth (registered trademark), etc.

[0043] The processing device 11 functions as a communication control unit 111, an acquisition unit 112, a display control unit 113, a detection unit 114, a selection unit 115, and an identification unit 116, for example, by reading and executing a control program PR1 from the storage device 12.

[0044] The communication control unit 111 causes the communication device 15 to transmit and receive various types of information to and from the image generation server 20 .

[0045] The acquisition unit 112 acquires a prompt that the user U[k] inputs using the input device 14 in order to cause the image generation server 20 to generate an image. The prompt acquired by the acquisition unit 112 is transmitted to the image generation server 20 by the communication device 25.

[0046] The acquisition unit 112 also acquires from the image generation server 20 a plurality of images generated by the image generation server 20. The plurality of images are images corresponding to each of the plurality of processes determined by executing the plurality of processes based on the prompt in chronological order, as described above.

[0047] The display control unit 113 causes the display 13 to display a user interface image UI that accepts operations by the user U[k].

[0048] The detection unit 114 detects an operation by the user U[k] on the user interface image UI.

[0049] The selection unit 115 selects one image from the plurality of images corresponding to each of the plurality of processes, based on the operation of the user U[k] detected by the detection unit 114. The display control unit 113 displays the one image selected by the selection unit 115 on the display 13. The one image is an example of a "first image."

[0050] 5A to 7B are diagrams showing examples of user interface images UI in this embodiment.

[0051] 5A and 5B show an example of a first user interface image UI1.

[0052] 5A and 5B, the first user interface image UI1 includes a first display area DA that displays an image acquired from the image generation server 20, a second display area SA that displays the number of steps corresponding to the image, and a slide bar SB. The slide bar SB includes a bar BR and a slider SL that the user U[k] slides on the bar BR when touching it with his / her finger. The slide bar SB is an example of a "controller."

[0053] 5A, the slider SL is located at the left end of the bar BR. In this state, the first display area DA displays an image GI1 corresponding to the first step of processing. The second display area SA displays "Step Number 1" to indicate that the step number corresponding to the image GI1 currently displayed in the first display area DA is the first.

[0054] 5A , in order to display an image GI corresponding to another step number in the first display area DA, the user U[k] touches the slider SL with his / her finger and slides the slider SL to the right on the bar BR. The detection unit 114 detects the above operation by the user U[k].

[0055] Based on the operation of user U[k] detected by detection unit 114, selection unit 115 acquires the distance that user U[k] has moved slider SL to the right on bar BR, and selects, from the multiple images GI acquired by acquisition unit 112, an image GI corresponding to the number of steps of processing according to the distance that slider SL has moved.

[0056] The display control unit 113 displays the image GI selected by the selection unit 115 in the first display area DA. The display control unit 113 also displays the number of steps corresponding to the image GI in the second display area SA.

[0057] 5B shows the first user interface image UI1 when the step number corresponding to the image GI selected by the selection unit 115 is the 12th step. In FIG. 5B, the slider SL is located to the right of the center of the bar BR. In this state, the first display area DA displays an image GI12 corresponding to the processing of the 12th step. Furthermore, the second display area SA displays "Step Number 12" to indicate that the step number corresponding to the image currently displayed in the first display area DA is the 12th step.

[0058] 6A and 6B show an example of a second user interface image UI2.

[0059] 6A and 6B , the second user interface image UI2 includes a first display area DA that displays an image GI acquired from the image generation server 20, a second display area SA that displays the number of steps corresponding to the image GI, and a dial DL. The dial DL includes an arrow AR that indicates the number of steps corresponding to the image GI currently displayed in the first display area DA. The dial DL is an example of an "operator."

[0060] 6A, the arrow AR is located at the top of the dial DL. In this state, the first display area DA displays an image GI1 corresponding to the first step of processing. The second display area SA displays "Step Number 1" to indicate that the step number corresponding to the image GI1 currently displayed in the first display area DA is the first.

[0061] 6A , the user U[k] touches the dial DL with his / her finger and rotates the dial DL to the right or left to display an image GI corresponding to another step number in the first display area DA. The detection unit 114 detects the above operation by the user U[k].

[0062] The selection unit 115 acquires the angle by which the user U[k] rotates the dial DL based on the operation of the user U[k] detected by the detection unit 114, and selects, from the multiple images GI acquired by the acquisition unit 112, an image GI corresponding to the number of processing steps according to the angle by which the dial DL has rotated.

[0063] The display control unit 113 displays the image GI selected by the selection unit 115 in the first display area DA. The display control unit 113 also displays the number of steps corresponding to the image GI in the second display area SA.

[0064] 6B shows the second user interface image UI2 when the step number corresponding to the image GI selected by the selection unit 115 is the 12th step. In FIG. 6B, the arrow AR points downward and to the right within the dial DL. In this state, the first display area DA displays the image GI12 corresponding to the processing of the 12th step. Furthermore, the second display area SA displays "Step Number 12" to indicate that the step number corresponding to the image GI currently displayed in the first display area DA is the 12th step.

[0065] 7A and 7B show an example of a third user interface image UI3.

[0066] As shown in Figures 7A and 7B, the third user interface image UI3 includes a first display area DA, which is an area for displaying an image GI obtained from the image generation server 20, and a second display area SA, which is an area for displaying the number of steps corresponding to the image GI.

[0067] 7A, the first display area DA displays an image GI1 corresponding to the first step of processing. The second display area SA displays "Step Number 1" to indicate that the step number corresponding to the image GI1 currently displayed in the first display area DA is the first.

[0068] In FIG. 7A, user U[k] touches the first display area DA with his / her finger and slides the finger to the right or left to display an image GI corresponding to another step number in the first display area DA.

[0069] The selection unit 115 acquires the direction and number of times that the user U[k] slid his / her finger based on the operation of the user U[k] detected by the detection unit 114, and selects, from the multiple image GIs acquired by the acquisition unit 112, an image GI corresponding to the number of steps of processing according to the direction and number of times that the user U[k] slid his / her finger.

[0070] The display control unit 113 displays the image GI selected by the selection unit 115 in the first display area DA. The display control unit 113 also displays the number of steps corresponding to the image GI in the second display area SA.

[0071] 6B shows the second user interface image UI2 when the step number corresponding to the image GI selected by the selection unit 115 is the 12th step. In Fig. 6B, the first display area DA displays an image GI12 corresponding to the processing of the 12th step. Furthermore, the second display area SA displays "Step Number 12" to indicate that the step number corresponding to the image GI12 currently displayed in the first display area DA is the 12th step.

[0072] In FIG. 4, the identifying unit 116 identifies a process in which the image corresponding to the process satisfies a first criterion, from among a plurality of processes that correspond one-to-one to a plurality of images GI.

[0073] In this embodiment, the identification unit 116 compares an image GI[a] corresponding to a first process among multiple processes executed in chronological order with an image GI[b] corresponding to a second process that occurs immediately after the first process. If the degree of change in image GI[b] relative to image GI[a] exceeds a threshold, the identification unit 116 identifies the second process. As described above, in the image generation server 20 according to this embodiment, noise is removed from the original image at each step. If image GI[b] is an image GI in which a significantly larger amount of noise has been removed from image GI[a], the identification unit 116 identifies the second process. Therefore, the identification unit 116 calculates the similarity between image GI[a] and image GI[b].

[0074] To calculate the similarity between image GI[a] and image GI[b], the determination unit 116 uses, as an example, the average pixel values ​​of both images GI. Note that, in this embodiment, "pixel value" refers to a luminance value. More specifically, the determination unit 116 calculates the average pixel values ​​of image GI[a] and image GI[b] for each of RGB. Then, the determination unit 116 calculates a three-dimensional Euclidean distance between the average pixel values ​​of image GI[a] and the average pixel values ​​of image GI[b] for each of RGB. The determination unit 116 may calculate the similarity between image GI[a] and image GI[b] based on the three-dimensional Euclidean distance. The determination unit 116 may calculate J levels of similarity by, for example, discretizing the three-dimensional Euclidean distance into J levels, where J is an integer greater than or equal to 2.

[0075] Alternatively, as another example, the determination unit 116 creates a histogram of image GI[a], where the pixel values ​​of image GI[a] are used as class values ​​and the frequencies are used as relative frequencies for each of RGB. Similarly, the determination unit 116 creates a histogram of image GI[b], where the pixel values ​​of image GI[b] are used as class values ​​and the frequencies are used as relative frequencies. The determination unit 116 calculates the square root of the sum of the squares of the differences in all classes between the histogram of image GI[a] and the histogram of image GI[b] for each of RGB, and adds up the square roots for all three RGB. The determination unit 116 may calculate the similarity between image GI[a] and image GI[b] based on the sum of the three roots. The determination unit 116 may calculate the J-level similarity by, for example, discretizing the sum of the three roots into J levels.

[0076] Alternatively, as another example, the determination unit 116 may adjust the resolution of the image GI[a] and the resolution of the image GI[b] to a certain value, and then calculate a correlation coefficient between the pixel values ​​of each of the RGB components of the image GI[a] and the image GI[b]. The determination unit 116 may calculate the similarity between the image GI[a] and the image GI[b] based on the correlation coefficient. The determination unit 116 may calculate the J-level similarity by, for example, discretizing the correlation coefficient into J levels.

[0077] Alternatively, as another example, the determination unit 116 performs image processing to adjust the resolution of image GI[a] and the resolution of image GI[b] to a fixed value. The determination unit 116 calculates the square root of the sum of squares of the differences in pixel values ​​at the same position for images GI[a] and GI[b] after the image processing. The determination unit 116 may calculate the similarity between images GI[a] and GI[b] based on the square root. The determination unit 116 may calculate the J-level similarity by, for example, discretizing the square root into J levels.

[0078] Alternatively, as another example, the identification unit 116 uses linear interpolation to change the width and height of image GI[a] to the power of two closest to the original size. Next, the identification unit 116 performs a discrete Fourier transform on the image with the changed width and height of image GI[a] to generate a converted image in which the frequency is the coordinate and the amplitude is the luminance. From the converted image in which the frequency is the coordinate and the amplitude is the luminance, the identification unit 116 generates a histogram corresponding to image GI[a], in which the classes are the amplitude and the frequency is the relative frequency. The identification unit 116 also uses a similar method for image GI[b] to generate a histogram corresponding to image GI[b]. The identification unit 116 calculates the square root of the sum of the squares of the differences in all classes from the histogram corresponding to image GI[a] and the histogram corresponding to image GI[b]. The determination unit 116 may calculate the similarity between the image GI[a] and the image GI[b] based on the square root. The determination unit 116 may calculate the J-level similarity by, for example, discretizing the square root into J levels.

[0079] Alternatively, as another example, the determination unit 116 uses linear interpolation to make the width and height of image GI[a] constant and a power of two. Next, the determination unit 116 performs a discrete Fourier transform on the image with the width and height of image GI[a] changed to generate a converted image with frequency as coordinates and amplitude as luminance. The determination unit 116 also uses a similar method for image GI[b] to generate a converted image with frequency as coordinates and amplitude as luminance. The determination unit 116 calculates the square root of the sum of the squares of the differences in luminance at the same position in both converted images. The determination unit 116 may calculate the similarity between image GI[a] and image GI[b] based on the square root. The determination unit 116 may calculate the similarity in J levels by, for example, discretizing the square root into J levels.

[0080] Alternatively, as another example, the identification unit 116 may calculate the similarity between the image GI[a] and the image GI[b] using both a feature based on an embedding vector in a deep learning model for image processing and a feature based on a hue histogram. Note that the "feature based on an embedding vector" generally refers to an image feature that places weight on the contours and semantic information of the image GI. Also, the "feature based on a hue histogram" generally refers to an image feature that places weight on color information.

[0081] Alternatively, as another example, the identification unit 116 may use the AKAZE method to extract feature points and feature amounts of each of the images GI[a] and GI[b], and then use feature point matching or the Bag of Visual Words method to calculate the similarity between the images GI[a] and GI[b].

[0082] Alternatively, as another example, the identification unit 116 may cause a trained model using deep learning to output the features of each of the images GI[a] and GI[b], and calculate the similarity between the images GI[a] and GI[b] so that the closer the output results are, the higher the similarity.

[0083] Alternatively, as another example, the specification unit 116 may calculate the similarity between the image GI[a] and the image GI[b] using a triplet loss method.

[0084] The identification unit 116, for example, uses any of the above methods to calculate the similarity between the image GI[a] and the image GI[b], and compares the image GI[a] corresponding to a first process among a plurality of processes executed in chronological order with the image GI[b] corresponding to a second process that occurs immediately after the first process. If the degree of change in the image GI[b] relative to the image GI[a] exceeds a threshold, the identification unit 116 identifies the second process.

[0085] The display control unit 113 displays a mark M corresponding to the process identified by the identification unit 116 in the user interface image UI.

[0086] In the user interface images UI exemplified in FIGS. 5A to 7B, a mark M is displayed indicating that the process identified by the identification unit 116 is the process of the 12th step.

[0087] 5A and 5B, a linear mark M that is perpendicular to the bar BR is displayed at a position on the bar BR that corresponds to the 12th step. The shape of the mark M is not limited to a linear shape and may be, for example, a circle, a triangle, or a rectangle.

[0088] 6A and 6B , among the multiple marks surrounding the dial DL, the mark corresponding to the 12th step is a larger and different colored mark M than the other marks. Note that the shape of the mark M is not limited to a circle and may be, for example, a triangle or a rectangle.

[0089] 7A and 7B , an image GI12 corresponding to the processing of the 12th step is displayed in the first display area DA, and at the same time, a mark M is displayed on the border of the second display area SA. The mark M is a line that is thicker than the border of the second display area SA itself and surrounds the second display area SA.

[0090] 1-2: Operation of First Embodiment FIG. 8 is a sequence diagram showing the operation of the image display system 1 according to this embodiment.

[0091] In step S1, the processing device 11 provided in the image display device 10[k] functions as the acquisition unit 112. The processing device 11 acquires a prompt input by the user U[k] using the input device .

[0092] In step S2, the processing device 11 provided in the image display device 10[k] functions as a communication control unit 111. The processing device 11 causes the communication device 15 of the image generation server 20 to transmit the prompt acquired in step S1. The processing device 21 provided in the image generation server 20 also functions as a communication control unit 211. The processing device 11 causes the communication device 25 to receive the prompt transmitted from the image display device 10[k].

[0093] In step S3, the processing device 21 included in the image generation server 20 functions as the generation unit 212. The processing device 21 acquires the prompt received in step S2. The processing device 21 then generates a plurality of images using the prompt. Specifically, the processing device 21 inputs the prompt into the image generation model IGM. The processing device 21 then acquires a plurality of images GI output from the image generation model IGM.

[0094] In step S4, the processing device 21 included in the image generation server 20 functions as a communication control unit 211. The processing device 21 causes the image display device 10[k] to transmit the multiple image GIs generated in step S3 to the communication device 25. The processing device 11 included in the image display device 10[k] functions as a communication control unit 111. The processing device 11 causes the communication device 15 to receive the multiple image GIs transmitted from the image generation server 20. The processing device 11 included in the image display device 10[k] functions as an acquisition unit 112. The processing device 11 acquires the multiple image GIs received by the communication device 15.

[0095] In step S5, the processing device 11 included in the image display device 10[k] functions as the identification unit 116. Among a plurality of processes that correspond one-to-one to a plurality of images GI, the processing device 11 identifies a process in which the quality of the image GI corresponding to the process satisfies a first criterion. In this embodiment, among a plurality of processes executed in chronological order, the processing device 11 compares an image GI[a] corresponding to a first process with an image GI[b] corresponding to a second process that occurs immediately after the first process. If the degree of change in image GI[b] relative to image GI[a] exceeds a threshold, the identification unit 116 identifies the second process.

[0096] In step S6, the processing device 11 functions as the display control unit 113. The processing device 11 displays a user interface image UI on the display 13. The user interface image UI includes a mark M corresponding to the process identified in step S5.

[0097] In step S7, the processing device 11 functions as the detection unit 114. The processing device 11 detects an operation by the user U[k] on the user interface image UI.

[0098] In step S8, the processing device 11 functions as the selection unit 115. Based on the operation of the user U[k] detected in step S7, the processing device 11 selects one image GI from among a plurality of image GIs that correspond one-to-one to a plurality of processes.

[0099] In step S9, the processing device 11 functions as the display control unit 113. The processing device 11 causes the display 13 to display the image GI selected in step S8.

[0100] The operations of steps S7 to S9 are repeated in response to the operation of the user U[k].

[0101] 1-3: Effects of the First Embodiment The image display device 10 according to this embodiment includes an acquisition unit 112, a display control unit 113, a detection unit 114, a selection unit 115, and an identification unit 116. The acquisition unit 112 acquires multiple images GIs that correspond one-to-one to the multiple processes determined by chronologically executing multiple processes based on a prompt. The acquisition unit 112 acquires multiple images GIs that correspond one-to-one to the multiple processes. The images GIs are displayed on the display 13 as a display device. The detection unit 114 detects the user U's operation on the user interface image UI. The selection unit 115 selects a first image from the multiple images GIs that correspond one-to-one to the multiple processes based on the user U's operation detected by the detection unit 114. The identification unit 116 identifies a process, among the multiple processes, whose corresponding image GI satisfies a first criterion. The display control unit 113 displays the first image on the display 13. The user interface image UI includes a mark M corresponding to the process identified by the identification unit 116.

[0102] Because the image display device 10 has the above configuration, when switching between multiple images GI to be displayed, the user U of the image display device 10 can intuitively grasp the change in quality of the image GI among the multiple images GI. More specifically, the image display device 10 identifies a process in which the image GI satisfies a first criterion. The user interface image UI displayed by the image display device 10 includes a mark M corresponding to the identified process. Therefore, the user U of the image display device 10 can intuitively grasp the change in the image GI.

[0103] In the image display device 10, the plurality of processes include a first process and a second process executed immediately after the first process. The first criterion is that the degree of change in the image corresponding to the second process relative to the image corresponding to the first process exceeds a threshold.

[0104] Because the image display device 10 has the above-mentioned configuration, a user U of the image display device 10 can grasp the image GI that has undergone a significant change among the multiple image GIs determined by executing multiple processes in chronological order.

[0105] Furthermore, in the image display device 10, the operation of the user U is an operation of moving the image of the operator included in the user interface image UI.

[0106] The image display device 10 has the above-described configuration, and thus the user U can switch the image GI to be displayed by moving the image of the control included in the user interface image UI.

[0107] Furthermore, in the image display device 10, the user interface image UI includes a first display area DA that displays the first image among the images corresponding to the plurality of processes. The operation by the user U is an operation on the first display area DA. In response to the operation by the user U, the display control unit 113 switches the first image displayed in the first display area DA to another image among the images corresponding to the plurality of processes.

[0108] The image display device 10 has the above configuration, so that the user U can switch the image GI to be displayed by swiping within the user interface image UI.

[0109] 2: Second Embodiment An image display system 1A according to a second embodiment will be described below with reference to Figures 9 to 12B. Note that, for the sake of simplicity, the following description will mainly focus on the differences between the image display system 1A according to this embodiment and the image display system 1 according to the first embodiment. Furthermore, among the components provided in the image display system 1A according to this embodiment, the same components as those provided in the image display system 1 according to the first embodiment will be designated by the same reference numerals, and a description of their functions may be omitted.

[0110] 2-1: Configuration of Second Embodiment 2-1-1: Overall Configuration The image display system 1A according to this embodiment includes image display devices 10A[1] to 10A[n] instead of the image display devices 10[1] to 10[n] included in the image display system 1 according to the first embodiment. The image display system 1A according to this embodiment also includes an image generation server 20A instead of the image generation server 20 included in the image display system 1 according to the first embodiment. In other respects, the overall configuration of the image display system 1A according to this embodiment is the same as the overall configuration of the image display system 1 according to the first embodiment, and therefore, the overall configuration of the image display system 1A according to this embodiment is not illustrated.

[0111] 2-1-2: Configuration of Image Generation Server The image generation server 20A includes a processing device 21A instead of the processing device 21 included in the image generation server 20 according to the first embodiment. The image generation server 20A also includes a storage device 22A instead of the storage device 22 included in the image generation server 20 according to the first embodiment. The processing device 21A includes a generation unit 212A instead of the generation unit 212 included in the processing device 21. The storage device 22A stores a control program PR2 and an image generation model IGMA instead of the control program PR2 and image generation model IGM stored in the storage device 22. In other respects, the configuration example of the image generation server 20A according to this embodiment is the same as the configuration example of the image generation server 20 according to the first embodiment, and therefore, the configuration example of the image generation server 20A according to this embodiment will not be illustrated.

[0112] The processing device 21A functions as a communication control unit 211 and a generation unit 212A, for example, by reading and executing a control program PR2A from the storage device 22A.

[0113] Similar to the generation unit 212, the generation unit 212A inputs a prompt acquired from the communication device 25 to the image generation model IGMA. The image generation model IGMA outputs an image based on the prompt. The generation unit 212A acquires the image output from the image generation model IGMA. Similar to the image generation model IGM, the image generation model IGMA is a well-known model such as DALL E 2 and Stable Diffusion.

[0114] 9 is a functional block diagram showing an example of the functions of the generation unit 212A according to this embodiment. The generation unit 212A reads the image generation model IGMA from the storage device 22A and executes it, thereby functioning as a batch size acquisition unit 701, a batch count acquisition unit 702, and an image generation unit 703.

[0115] The batch size acquisition unit 701 acquires a batch size. The "batch size" is the number of images GI that the image generation unit 703 generates at one time. For example, a user U[k] inputs a batch size to the image display device 10A[k] using the input device 14. The batch size acquisition unit 701 acquires the batch size input by the user U[k] from the image display device 10A[k]. Alternatively, the batch size acquisition unit 701 may acquire a predetermined batch size that is pre-stored in the storage device 22A.

[0116] The batch count acquisition unit 702 acquires the batch count. The "batch count" is the number of times the image generation unit 703 generates the number of images indicated by the batch size. The image generation unit 703 executes processing for the batch count in chronological order. As a result, the image generation unit 703 generates a number of images equal to the product of the batch size and the batch count. These images are different from one another. For example, a user U[k] inputs the batch count to the image display device 10A[k] using the input device 14. The batch count acquisition unit 702 acquires the batch count input by the user U[k] from the image display device 10A[k]. Alternatively, the batch count acquisition unit 702 may acquire a predetermined batch count that is pre-stored in the storage device 22A.

[0117] As described above, the image generation unit 703 generates the number of images GI equal to the product of the batch size and the number of batches. The image generation unit 703 outputs the generated images GI in time series.

[0118] The communication control unit 211 causes the image display device 10A[k] to transmit, via the communication device 25, a plurality of images GI that have been generated by the generation unit 212A and correspond one-to-one to a plurality of processes.

[0119] 2-1-3: Configuration of Image Display Device The image display device 10A includes a processing device 11A instead of the processing device 11 included in the image display device 10 according to the first embodiment. The image display device 10A also includes a storage device 12A instead of the storage device 12 included in the image display device 10 according to the first embodiment. The processing device 11A includes a specification unit 116A instead of the specification unit 116 included in the processing device 11. The storage device 12A stores a control program PR1A instead of the control program PR1 stored in the storage device 12. In other respects, the configuration example of the image display device 10A according to this embodiment is the same as the configuration example of the image display device 10 according to the first embodiment, and therefore, the configuration example of the image display device 10A according to this embodiment will not be illustrated.

[0120] The processing device 11A functions as a communication control unit 111, an acquisition unit 112, a display control unit 113, a detection unit 114, a selection unit 115, and an identification unit 116A, for example, by reading and executing a control program PR1A from the storage device 12A.

[0121] Similar to the identification unit 116, the identification unit 116A identifies a process, among a plurality of processes that correspond one-to-one to a plurality of image GIs, in which the quality of the image corresponding to the process satisfies a first criterion. However, unlike the identification unit 116, in this embodiment, the identification unit 116A identifies a process corresponding to the image GI with the highest quality among the plurality of image GIs. For this reason, the identification unit 116A calculates a value indicating the quality of each of the plurality of image GIs. In this embodiment, "quality" refers to the degree to which the image GI corresponding to the prompt resembles an image captured in the real world.

[0122] To calculate the quality of the image GI, the determination unit 116A uses, for example, FID (Fréchet Inception Distance). Specifically, the determination unit 116A extracts feature quantities of the image GI acquired from the image generation server 20 and calculates the mean vector and covariance matrix of the feature quantities. The determination unit 116A also extracts feature quantities of an image captured in real space and calculates the mean vector and covariance matrix of the feature quantities. For example, if the image GI acquired from the image generation server 20 is an image generated based on the prompt "penguin," the "image captured in real space" is an image of a penguin captured in real space. Furthermore, the determination unit 116A calculates the similarity between the image GI and the real image by calculating the sum of the Euclidean distance of the mean vector and the Frobenius distance of the covariance matrix for both feature quantities as the FID. It can be said that the higher the FID value, the higher the quality of the image GI.

[0123] The determination unit 116A may also use a CLIP score, as another example, to calculate the quality of the image GI. Specifically, the determination unit 116A converts the prompt used to generate the image GI and the image GI into feature vectors. The determination unit 116A also calculates the cosine similarity between the feature vectors as the CLIP score. The higher the CLIP score, the higher the quality of the image GI.

[0124] The identification unit 116A may also use an Inception Score (IS) as another example to calculate the quality of an image. Specifically, the identification unit 116A classifies the image GI acquired from the image generation server 20 using an Inception network, and determines that the higher the entropy of the classification result, the higher the quality of the image. More specifically, the identification unit 116A determines that the higher the quality of an image GI, the more frequently one of multiple labels used for classification is used and the less frequently the other labels are used.

[0125] The display control unit 113 displays a mark M corresponding to the process identified by the identification unit 116A in the user interface image UI.

[0126] 10A to 12B are diagrams showing examples of a user interface image UIA in this embodiment.

[0127] 10A and 10B show an example of a first user interface image UIA1.

[0128] 5A and 5B, the first user interface image UIA1 according to the present embodiment differs from the first user interface image UI1 according to the first embodiment in that the number of generated images, which indicates the number of images generated, is displayed in the second display area SA instead of the number of steps. In FIG. 5B, as an example, the eighth generated image GI8 is displayed in the first display area DA.

[0129] The same applies to the example of the second user interface image UIA2 shown in FIGS. 11A and 11B and the example of the third user interface image UIA3 shown in FIGS. 12A and 12B.

[0130] Furthermore, in the user interface image UIA exemplified in FIGS. 10A to 12B, a mark M is displayed indicating that the process identified by the identification unit 116A is the process that generated the highest quality image GI.

[0131] 2-2: Operation of the Second Embodiment When comparing the operation of the image display system 1A according to this embodiment with the operation of the image display system 1 according to the first embodiment shown in FIG. 8, the operation is the same except for the details of the operation of step S5, and therefore is not shown in the figure.

[0132] In step S5 of this embodiment, the processing device 11A included in the image display device 10A[k] functions as the identification unit 116A. The processing device 11A identifies, from among a plurality of processes that correspond one-to-one to a plurality of images GI, a process in which the quality of the image GI corresponding to the process satisfies a first criterion. In this embodiment, the processing device 11A identifies, from among the plurality of images GI acquired in step S4, a process corresponding to the image GI with the highest quality.

[0133] 2-3: Effects of the Second Embodiment In the image display device 10A according to this embodiment, the first criterion is that the corresponding image has the highest quality among the multiple images that correspond one-to-one to the multiple processes.

[0134] By having the above-described configuration, the image display device 10A allows a user U of the image display device 10A to grasp the image GI with the highest quality among multiple image GIs determined by executing multiple processes in chronological order.

[0135] In addition, in the image display device 10A, the quality indicates the degree of similarity between the corresponding image and an image captured in real space corresponding to the prompt.

[0136] By having the above-described configuration, the image display device 10A allows a user U of the image display device 10A to grasp the image GI that is most similar to the image captured in real space among multiple image GIs determined by executing multiple processes in chronological order.

[0137] 3: Modifications The present disclosure is not limited to the above-described exemplary embodiments. Specific modifications are exemplified below. Two or more modifications selected from the following examples may be combined. Furthermore, the above-described embodiments and the following modifications may be combined in any manner as long as they are not mutually contradictory.

[0138] 3-1: Modification 1 In the first embodiment, the image display device 10 includes a communication control unit 111, an acquisition unit 112, a display control unit 113, a detection unit 114, a selection unit 115, and an identification unit 116. However, the image generation server 20, rather than the image display device 10, may include at least one of these components.

[0139] For example, the image generation server 20 may include the display control unit 113, so that the image generation server 20 can cause the image display device 10 to display the user interface image UI.

[0140] Alternatively, the image generation server 20 may be provided with the detection unit 114 , so that the image generation server 20 detects the operation of the user U on the user interface image UI in the image display device 10 .

[0141] Alternatively, the image generation server 20 may be provided with a selection unit 115, so that the image generation server 20 can select one image GI from among image GIs corresponding to multiple processes based on the operation of the user U.

[0142] Alternatively, the image generation server 20 may be provided with the identification unit 116, so that the image generation server 20 may identify, from among a plurality of processes, a process whose corresponding image quality satisfies the first criterion.

[0143] The same applies to the second embodiment.

[0144] 3-2: Modification 2 When the image display device 10 displays an image GI corresponding to the processing identified by the identification unit 116, the image display device 10 may output a sound effect instead of displaying the mark M. Alternatively, the image display device 10 may display the mark M and output a sound effect.

[0145] 3-3: Modification 3 As described above, in the first embodiment, the criterion for the process identified by the identification unit 116 to be satisfied is that the degree of change in the image corresponding to the second process relative to the image corresponding to the first process exceeds a threshold value, between the first process and the second process executed immediately after the first process.

[0146] However, in the first embodiment, the criterion for the process identified by the identification unit 116 to satisfy may be that the process corresponds to the highest quality image GI among a plurality of image GIs, as in the second embodiment.

[0147] Alternatively, the criteria for a process identified by the identification unit 116 to be satisfied may be that the degree of change in the image GI corresponding to the second process relative to the image GI corresponding to the first process exceeds a threshold, and that the process corresponds to the image GI of the highest quality among the image GIs that exceed the threshold.

[0148] 3-4: Modification 4 In the first embodiment, the multiple images GI generated by the generation unit 212 included in the image generation server 20 were transmitted to the image display device 10. The multiple images GI may or may not be stored in the storage device 22 included in the image generation server 20. Furthermore, the image GI transmitted to the image display device 10 may or may not be stored in the storage device 12 included in the image display device 10. The same applies to the second embodiment.

[0149] 4: Others (1) In the above-described embodiment, storage device 12, storage device 12A, storage device 22, and storage device 22A are exemplified by ROM and RAM, but they may also be flexible disks, magneto-optical disks (e.g., compact disks, digital versatile disks, Blu-ray (registered trademark) disks), smart cards, flash memory devices (e.g., cards, sticks, key drives), CD-ROMs (Compact Disc-ROMs), registers, removable disks, hard disks, floppy (registered trademark) disks, magnetic strips, databases, servers, or other suitable storage media. The program may also be transmitted from a network via a telecommunications line. The program may also be transmitted from a communications network (NET) via a telecommunications line.

[0150] (2) In the above-described embodiments, the described information, signals, etc. may be represented using any of a variety of different technologies. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any combination thereof.

[0151] (3) In the above-described embodiment, input and output information may be stored in a specific location (for example, a memory) or may be managed using a management table. Input and output information may be overwritten, updated, or added to. Output information may be deleted. Input information may be transmitted to another device.

[0152] (4) In the above-described embodiment, the determination may be made based on a value (0 or 1) represented using one bit, a Boolean value (true or false), or a comparison of numerical values ​​(e.g., comparison with a predetermined value).

[0153] (5) The order of the exemplary procedures, sequences, flowcharts, etc. illustrated in the above-described embodiments may be rearranged unless inconsistent. For example, the methods described in this disclosure present elements of various steps using an example order, and are not limited to the particular order presented.

[0154] (6) Each function illustrated in Figures 1 to 12B is realized by any combination of hardware and / or software. Furthermore, the method of realizing each functional block is not particularly limited. That is, each functional block may be realized using a single device that is physically or logically coupled, or may be realized using two or more physically or logically separated devices that are directly or indirectly connected (e.g., wired, wireless, etc.) and these multiple devices. A functional block may be realized by combining software with the single device or the multiple devices.

[0155] (7) The programs exemplified in the above-described embodiments should be broadly construed to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, threads of execution, procedures, functions, etc., regardless of whether they are called software, firmware, middleware, microcode, hardware description language, or by other names.

[0156] Software, instructions, information, etc. may also be transmitted or received over a transmission medium. For example, if software is transmitted from a website, server, or other remote source using wired technologies (such as coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL)), and / or wireless technologies (such as infrared, microwave), then these wired and / or wireless technologies are included within the definition of transmission media.

[0157] (8) In each of the foregoing embodiments, the terms "system" and "network" are used interchangeably.

[0158] (9) The information, parameters, etc. described in this disclosure may be expressed using absolute values, relative values ​​from a predetermined value, or corresponding other information.

[0159] (10) In the above-described embodiments, the image display device 10, the image display device 10A, the image generation server 20, and the image generation server 20A may be mobile stations (MS). A mobile station may also be referred to by those skilled in the art as a subscriber station, a mobile unit, a subscriber unit, a wireless unit, a remote unit, a mobile device, a wireless device, a wireless communication device, a remote device, a mobile subscriber station, an access terminal, a mobile terminal, a wireless terminal, a remote terminal, a handset, a user agent, a mobile client, a client, or some other appropriate term. In addition, in this disclosure, terms such as "mobile station," "user terminal," "user equipment (UE)," and "terminal" may be used interchangeably.

[0160] (11) In the above-described embodiments, the terms "connected," "coupled," or any variations thereof refer to any direct or indirect connection or coupling between two or more elements, and may include the presence of one or more intermediate elements between two elements that are "connected" or "coupled" to each other. The coupling or connection between elements may be a physical coupling or connection, a logical coupling or connection, or a combination thereof. For example, "connected" may be read as "access." As used in this disclosure, two elements may be considered to be "connected" or "coupled" to each other using at least one of one or more wires, cables, and printed electrical connections, as well as electromagnetic energy having wavelengths in the radio frequency range, microwave range, and optical (both visible and invisible) range, as some non-limiting and non-exhaustive examples.

[0161] (12) In the above embodiments, the phrase "based on" does not mean "based only on," unless otherwise specified. In other words, the phrase "based on" means both "based only on" and "based at least on."

[0162] (13) As used in this disclosure, the terms "determining" and "determining" may encompass a wide variety of actions. "Determining" and "determining" may include, for example, judging, calculating, computing, processing, deriving, investigating, looking up, searching, inquiring (e.g., searching a table, database, or other data structure), and ascertaining something that is considered to be a "determining." Also, "determining" and "determining" may include receiving (e.g., receiving information), transmitting (e.g., sending information), input, output, accessing (e.g., accessing data in memory), and so on. Furthermore, "judgment" and "decision" can include regarding resolving, selecting, choosing, establishing, comparing, etc. as having been "judged" or "decided." In other words, "judgment" and "decision" can include regarding some action as having been "judged" or "decided." Furthermore, "judgment (decision)" can be interpreted as "assuming," "expecting," "considering," etc.

[0163] (14) In the above embodiments, when the terms "include," "including," and variations thereof are used, these terms are intended to be inclusive, similar to the term "comprising." Furthermore, the term "or," as used in this disclosure, is not intended to be an exclusive or.

[0164] (15) In this disclosure, where articles are added by translation, such as a, an, and the in English, this disclosure may include the nouns following these articles being plural.

[0165] (16) In this disclosure, the term "A and B are different" may mean "A and B are different from each other." The term may also mean "A and B are each different from C." Terms such as "separate" and "combined" may also be interpreted in the same way as "different."

[0166] (17) Each aspect / embodiment described in this disclosure may be used alone, in combination, or switched depending on the implementation. Notification of predetermined information (e.g., notification that "X is true") is not limited to explicit notification, but may be implicit (e.g., not notifying the predetermined information).

[0167] Although the present disclosure has been described in detail above, it is clear to those skilled in the art that the present disclosure is not limited to the embodiments described herein. The present disclosure can be implemented in modified and altered forms without departing from the spirit and scope of the present disclosure as defined by the claims. Therefore, the description of the present disclosure is intended to be illustrative and does not have any limiting meaning on the present disclosure.

[0168] 1, 1A... image display system, 10, 10A... image display device, 11, 11A... processing device, 12, 12A... storage device, 13... display, 14... input device, 15... communication device, 20, 20A... image generation server, 21, 21A... processing device, 22, 22A... storage device, 23... display, 24... input device, 25... communication device, 111... communication control unit, 112... acquisition unit, 113... display control unit, 114... detection unit, 115... selection unit, 116, 116A... identification unit, 211... communication control unit, 212, 212A... generation unit, 501... prompt acquisition unit, 502... Original image acquisition unit, 503...correct image prediction unit, 504...predicted noise generation unit, 505...noise removal unit, 506...step number acquisition unit, 701...batch size acquisition unit, 702...batch count acquisition unit, 703...image generation unit, AR...arrow, BR...bar, DA...first display area, DL...dial, GI...image, IGM, IGMA...image generation model, M...mark, NET...communication network, PR1, PR1A, PR2, PR2A...control program, SA...second display area, SB...slide bar, SL...slider, U...user, UI, UIA...user interface image

Claims

1. An image display device comprising: an acquisition unit that acquires a plurality of images that correspond one-to-one to a plurality of processes determined by executing a plurality of processes based on prompts in chronological order; a display control unit that causes a display device to display a user interface image that accepts user operations; a detection unit that detects the user's operations on the user interface image; a selection unit that selects a first image from a plurality of images that correspond one-to-one to the plurality of processes based on the user's operations detected by the detection unit; and an identification unit that identifies a process from the plurality of processes whose corresponding image satisfies a first criterion; wherein the display control unit causes the display device to display the first image, and the user interface image includes a mark corresponding to the process identified by the identification unit.

2. An image display device as described in claim 1, wherein the plurality of processes include a first process and a second process executed immediately after the first process, and the first criterion is that the degree of change in the image corresponding to the second process relative to the image corresponding to the first process exceeds a threshold.

3. An image display device according to claim 1, wherein the first criterion is that the corresponding image has the highest quality among a plurality of images that correspond one-to-one to the plurality of processes.

4. The image display device according to claim 3, wherein the quality indicates the degree of similarity between the corresponding image and an image captured in real space corresponding to the prompt.

5. The image display device according to claim 1, wherein the user's operation is an operation of moving an image of a control included in the user interface image.

6. The image display device of claim 1, wherein the user interface image includes an area for displaying the first image among the images that correspond one-to-one to the multiple processes, the user's operation is an operation on the area, and the display control unit switches the first image displayed in the area to another image among the multiple images that correspond one-to-one to the multiple processes in response to the user's operation.

Images