Control device and computer program

Abstract

To provide a technique capable of suppressing reduction in efficiency for searching for a desired image.SOLUTION: A control device includes: a class selection display control unit that causes a display device to display a class selection image; and an image display control unit that causes the display device to display a first image group as an image group having a first reference image associated with a first class and a plurality of first determined images, in which the image display control unit causes the display device to display a second image group as an image group having a plurality of second determined images as a plurality of determined images determined according to a new reference image, and causes the display device to display the second image group when a second class is selected instead of the first class via the class selection image.SELECTED DRAWING: Figure 8

Description

[Technical Field]

[0001] The present disclosure relates to a technique for displaying an image on a display device. [Background technology]

[0002] BACKGROUND ART Conventionally, there is known a technique for automatically generating an image desired by a user based on an image selected in response to a user operation and displaying the image on a display device (Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2023-11668 Summary of the Invention [Problem to be solved by the invention]

[0004] In conventional technology, images are mapped into a predetermined space, and some or all of the images present within a search area centered on a selected reference image in the predetermined space are displayed on a display device as candidate images. Furthermore, in conventional technology, a user selects a reference image from thumbnails of images previously selected as reference images or from among candidate images. However, in conventional technology, it may be difficult for a user to determine in which area of ​​the predetermined space the desired image is located. In this case, the efficiency of searching for the desired image may be reduced. [Means for solving the problem]

[0005] According to a first aspect of the present disclosure, there is provided a control device for controlling image display, the control device including: a class selection display control unit that causes a display device to display a class selection image that accepts selection of an image class to be specified from among a plurality of image classes; and an image display control unit that causes the display device to display a first image group as a group of images including a first reference image as a reference image associated with a first class as the selected image class and a plurality of first determined images as a plurality of determined images determined according to the first reference image, wherein the image display control unit causes the display device to display a second image group as the group of images including the selected image as a new reference image and a plurality of second determined images as the plurality of determined images determined according to the new reference image when one of the images in the first image group is selected; and when a second class as the image class is selected instead of the first class via the class selection image, the image display control unit causes the display device to display a second image group including the second reference image as the reference image associated with the selected second class and a plurality of second determined images determined according to the second reference image.

[0006] According to a second aspect of the present disclosure, there is provided a computer program for controlling image display. The computer program includes: (a) a function for causing a display device to display a class selection image that accepts selection of an image class from among a plurality of image classes; and (b) a function for causing the display device to display a first image group as an image group including a first reference image as a reference image associated with a first class as the selected image class and a plurality of first determined images as a plurality of determined images determined in accordance with the first reference image, wherein function (b) causes the display device to display, when one of the images in the first image group is selected, a second image group including the selected image as a new reference image and a plurality of second determined images as the plurality of determined images determined in accordance with the new reference image; and, when a second class as the image class is selected instead of the first class via the class selection image, function (b) causes the display device to display a second image group including the second reference image as the reference image associated with the selected second class and a plurality of second determined images determined in accordance with the second reference image. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram showing a display system according to a first embodiment. [Figure 2] Internal block diagram of the server. [Figure 3] FIG. 1 is a diagram showing a group of image sets. [Figure 4] FIG. 2 is a diagram schematically showing mapping information. [Figure 5] FIG. [Figure 6] 10 is a flowchart showing a display process generated by the display system. [Figure 7] 10 is a flowchart showing a generation process of a condition input image. [Figure 8] FIG. 10 is a diagram showing a screen including a condition input image displayed on a display device. [Figure 9] FIG. 7 is a diagram for explaining step S35 in FIG. 6. [Figure 10] 10A and 10B are diagrams for explaining a condition input image and an image group. [Figure 11] FIG. 7 is a diagram for explaining step S5 in FIG. 6. [Figure 12] 10A and 10B are diagrams for explaining a condition input image and an image group displayed on a display device. [Figure 13] FIG. 7 is a diagram for explaining step S5 in FIG. 6, which is executed in response to request information. [Figure 14] FIG. 4 is a diagram for explaining a mapping image displayed on a display device. [Figure 15] FIG. 10 is a diagram showing a condition input image in the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] A. First embodiment: 1 is a diagram showing a display system 5 in a first embodiment. The display system 5 includes a server 10, a control device 40 as a display control device that controls the display of images, a display device 80, and an input device 99. The server 10 and the control device 40 are connected to each other via the Internet INT or the like so as to be able to communicate data with each other. The control device 40, the display device 80, and the input device 99 are also connected to each other by wire or wirelessly so as to be able to communicate data with each other.

[0009] The server 10 generates at least one image information IFM in response to request information RI for image generation received from the control device 40. The server 10 transmits response information AI including the generated image information IFM to the control device 40. The control device 40 is an electronic computer such as a personal computer. The control device 40 displays the image included in the response information AI received from the server 10 on the display device 80. The display device 80 displays various information such as images. The display device 80 is, for example, an LCD monitor. The input device 99 is, for example, a device that inputs data to the control device 40. The input device 99 is, for example, a keyboard or a mouse. Note that in this embodiment, the server 10 and the control device 40 are separate devices, but the server 10 and the control device 40 may be configured as the same device.

[0010] FIG. 2 is an internal block diagram of the server 10. The server 10 includes a server processor 12, a server storage unit 20, and a server interface 28. The server processor 12 functions as a generation unit 14 by executing various programs stored in the server storage unit 20. Note that some of the functions executed by the server processor 12 may be implemented by a hardware circuit. Here, in this disclosure, the term "processor" encompasses a CPU and a GPU. The generation unit 14 generates response information AI including an image corresponding to request information RI received from the control device 40. The generation unit 14 generates an image corresponding to the request information RI using a generative model 22 and an image set group 24, which will be described later. The server interface 28 is an interface for data communication with an external device such as the control device 40.

[0011] The server storage unit 20 includes a generative model 22 and an image set group 24. The generative model 22 is a model that generates image information IFM. The generative model 22 uses a known machine learning algorithm, such as a Boltzmann machine, an Encoder-Decoder, a GAN, a Variational Autoencoder (VAE), or StyleTransfer. In this embodiment, the generative model 22 is a model that uses a VAE algorithm. That is, the generative model 22 includes an encoder unit and a decoder unit. The encoder unit dimensionally compresses the input image information IFM, i.e., extracts features of the image information IFM and represents them using a latent variable Z. The decoder unit generates and outputs image information IFM from the latent variable Z. In this embodiment, the generative model 22 dimensionally compresses the image information IFM (described later) onto a latent space, which is a predetermined space represented by two-dimensional coordinates Z(x, y). The generative model 22 is trained in advance so that the similarity between the input image information IFM and the output image information IFM is equal to or greater than a predetermined reference value. In other embodiments, the generative model 22 is not limited to a model using a VAE algorithm, and may be, for example, a diffusion model.

[0012] The image set group 24 includes image information IFM used in training the generative model 22. Note that in other embodiments, the training set group 24 may also include image information IFM that was not used in training the generative model 22. FIG. 3 is a diagram showing the image set group 24. The image set group 24 includes a plurality of image sets, each having an image identifier ID, an image IL, an image class CL for classifying the image IL, and position information LT. The image IL, the image class CL, and the position information LT are associated with the image identifier ID. Here, the image IL and the image class CL constitute the image information IFM.

[0013] The image IL is an image acquired by various methods, such as an image captured by an imaging device or an image generated using image generation software. The image IL in this embodiment is an image of an animal, specifically, an image of a fish, a cat, or a dog.

[0014] The image class CL is an element for distinguishing between multiple types of images IL. The image class CL is associated with each image IL by the user or the like. In this embodiment, the image class CL is information indicating the types of animals, such as "fish," "cat," and "dog." The image class CL does not have to be character information, and may be, for example, a sequence of numbers, as long as it can distinguish between multiple types of images IL.

[0015] The position information LT indicates the position when the image information IFM is represented in a predetermined space by the generative model 22. In this embodiment, since the algorithm of the generative model 22 uses VAE, the position information LT is represented by the coordinate Z in the latent space. As described above, in this embodiment, in addition to the image IL, the image class CL associated with the image IL is also an object to be represented in a predetermined space by the generative model 22. In other words, multiple pieces of image information IFM are mapped in a predetermined space according to the similarity between the unit sets, with the image IL and the image class CL being one unit set.

[0016] FIG. 4 is a diagram illustrating the mapping information Mp. The mapping information Mp is information in which multiple pieces of image information IFM are represented by the number of dimensions in a latent space, which is a predetermined space, and are mapped onto the predetermined space. For ease of understanding, the mapping information Mp arranges images IL in the latent space. However, image identifiers ID may also be plotted in the latent space. Furthermore, as shown in FIG. 3, the mapping information Mp only needs to associate position information LT with image information IFM. In other words, the image set group 24 shown in FIG. 3 can be said to constitute the mapping information Mp. In this embodiment, the closer the distance between coordinate points in the predetermined space indicated by the position information LT, the higher the similarity between the unit sets. For example, even if the images IL of two unit sets are somewhat similar to each other, if the image classes CL are different, the two pieces of image information IFM will be arranged at positions somewhat apart in the predetermined space.

[0017] The image information IFM converted into the latent variable Z by the encoder unit of the generative model 22 includes an image IL and an image class CL associated with the image IL, as shown in Fig. 3. Therefore, as shown in Fig. 4, the image information IFM is arranged in the latent space in a region that is somewhat concentrated for each image class.

[0018] 5 is an internal block diagram of the control device 40. The control device 40 includes a device processor 42, a device storage unit 50, and a device interface 58. The device processor 42 executes various programs stored in the device storage unit 50 to function as an acquisition unit 43 and a display control unit 44. Note that some of the functions executed by the device processor 42 may be realized by hardware circuits. The device storage unit 50 stores various programs executed by the device processor 42. The device interface 58 is an interface for data communication with external devices such as the server 10.

[0019] The acquisition unit 43 generates request information RI according to instructions related to image generation received via the input device 99, and transmits the request information RI to the server 10 via the device interface 58. The acquisition unit 43 also acquires response information AI transmitted from the server 10 in response to the request information RI via the device interface 58. The response information AI is stored in the device storage unit 50.

[0020] 1 to display various information on the display device 80. The display control unit 44 has a class selection display control unit 46, an image display control unit 47, an area display control unit 48, and a mapping display control unit 49.

[0021] The class selection display control unit 46 generates a class selection image IM1 that accepts the selection of a specified image class CL from multiple image classes CL, and displays the image on the display device 80. The image display control unit 47 accepts instructions from a user and displays an image group IG including multiple images corresponding to the instructions on the display device 80. The user's instructions are given via the class selection image IM1 or the like. The region display control unit 48 generates a region selection image IM2 for determining the size of the search region SR in a latent space, which is a predetermined space that defines the image information IFM, and displays the image on the display device 80. The mapping display control unit 49 displays a mapping selection image IM3 on the display device 80 to display a mapping image IMp on the display device 80. The mapping information Mp is information in which multiple pieces of image information IFM, represented by the number of dimensions of the latent space, which is a predetermined space, are mapped onto the latent space. The mapping information Mp may be generated by the mapping display control unit 49 or by the server 10. Details of the various images IM1 to IM4 displayed on the display device 80 will be described later.

[0022] Fig. 6 is a flowchart showing the display processing generated by the display system 5. Fig. 7 is a flowchart showing the generation processing of the condition input image in step S10 of Fig. 6. Fig. 8 is a diagram showing a screen including the condition input image IM displayed on the display device 80. Fig. 9 is a diagram for explaining step S35 of Fig. 6. The display processing is started when the device processor 42 receives an instruction to start processing from the user via the input device 99.

[0023] As shown in FIG. 6, in step S10, the control device 40 generates a condition input image IM and displays it on the display device 80. As shown in FIG. 7, in generating and displaying the condition input image IM, the class selection display control unit 46 generates a class selection image IM1 in step S12. Then, in step S14, the region display control unit 48 generates a region selection image IM2. Then, in step S15, the mapping display control unit 49 generates a mapping selection image IM3. Note that the order of steps S12, S14, and S15 is not limited to this. Next, in step S16, the display control unit 44 displays the condition input image IM, which includes the class selection image IM1, the region selection image IM2, the mapping selection image IM3, and a reception image IM4 (described later), on the display device 80.

[0024] 8, the condition input image IM includes a class selection image IM1, a region selection image IM2, a mapping selection image IM3, and a reception image IM4. The reception image IM4 is generated by the display control unit 44 in any one of step S12, step S14, and step S15.

[0025] The class selection image IM1 is an image for accepting an image class CL designated by the user from among a plurality of image classes CL. The class selection image IM1 allows the user to select one of a plurality of image classes CL prepared in advance using a pull-down menu. The class selection image IM includes the characters "Dog" indicating the image class CL of "dog," "Fish" indicating the image class CL of "fish," and "Cat" indicating the image class CL of "cat." Note that the class selection image IM1 is not limited to the above. For example, the class selection image IM1 may display a list of images corresponding to a plurality of image classes CL so that they can be selected.

[0026] The region selection image IM2 is an image for determining the size of the search region SR in a predetermined space. Image information IFM that the user wants to display on the display device 80 as an image group IG is located in the predetermined space. The region selection image IM2 has a bar image IM2a that indicates the size of the search region SR, and a point image IM2b that determines the size of the search region SR by moving the bar image IM2a. The user determines the search region SR, for example, by moving the point image IM2b on the bar image IM2a with a cursor displayed on the display device 80.

[0027] The mapping selection image IM3 is an image for displaying the mapping information Mp on the display device 80. When the user selects the mapping selection image IM3 with a cursor or the like, the mapping display control unit 49 displays the mapping image IMp generated using the mapping information Mp on the display device 80.

[0028] The reception image IM4 is an image for receiving the image class CL and search region SR specified by the class selection image IM1 and the region selection image IM2 as requested information RI. For example, when the reception image IM4 is selected with a cursor or the like, the requested information RI is generated by the acquisition unit 43. The image class CL specified by the class selection image IM1 is also referred to as a specified image class CLs. The search region SR specified by the region selection image IM2 is also referred to as a specified search region SRs.

[0029] As described above, when the received image IM4 is selected, the acquisition unit 43 generates request information RI including the designated image class CLs and the designated search region SRs in step S30 shown in Fig. 6. The acquisition unit 43 transmits the generated request information RI to the server 10.

[0030] The generating unit 14 of the server 10 that has acquired the request information RI generates response information AI according to the request information RI in step S35. The response information AI generated by the generating unit 14 is transmitted to the control device 40.

[0031] As shown in FIG. 9, in step S35, the generation unit 14 of the server processor 12 selects a reference image ILs. Specifically, the generation unit 14 first extracts image information IFMs associated with designated image classes CLs from among the image information IFMs included in the image set group 24 shown in FIG. 2. The generation unit 14 then selects one image information IFM from the extracted image information IFMs as representative image information IFMs, and selects an image IL included in the selected representative image information IFMs as the reference image ILs. For example, the generation unit 14 selects image information IMF located at a representative coordinate point in each piece of position information LT in the extracted image information IFMs as the representative image information IFMs. The representative coordinate point may be, for example, a coordinate point closest to the average value of each coordinate in a predetermined space, which is the position information in the extracted image information IFM, a coordinate point predetermined for each image class CL, or a coordinate point set by the user. In the example shown in FIG. 9, the image information IFM indicated by the latent variable Z(x4, y4) indicating the coordinate point is selected as the representative image information IFMs.

[0032] The generation unit 14 determines a predetermined number N of image information IFMs from among the image information IFMs corresponding to the latent variables Z located within a search region SR centered on the representative image information IFMs in a predetermined space. In this embodiment, the predetermined number is "8," but is not limited to this. The predetermined number N may be configurable by a user or the like. As shown in FIG. 9 , the search region SR is a circle centered on the coordinate point of the latent variable Z corresponding to the representative image information IFMs in a latent space, which is a predetermined space. This circle is set according to the size of the search region SR, and the larger the search region SR, the larger the radius of the circle. The generation unit 14 determines the predetermined number N of latent variables Z within the search region SR so as to satisfy a determination condition. For example, the generation unit 14 may determine the latent variables Z so as to satisfy a determination condition that the variance of the coordinate points of the eight latent variables Z is maximized. For example, the generation unit 14 may extract latent variables Z so as to satisfy a determination condition that a predetermined number N of latent variables Z are determined, each representing a coordinate point located in a different direction from the representative coordinate point in the latent space. The different directions may be set so that the angles formed by adjacent directions are equal. Furthermore, the generation unit 14 may generate new latent variables Z when there are no latent variables Z to extract or when the number of latent variables Z included in the image set group 24 is insufficient. In this case, the generation unit 14 generates additional image information IFM by inputting the newly generated latent variables Z to the decoder unit of the generative model 22. The generation unit 14 determines the image information IFM corresponding to the determined latent variable Z as the determined image information IMFs. When the image set group 24 already includes image information IMF corresponding to the determined latent variable Z, the generation unit 14 determines the image information IMF of the image set group 24 as the determined image information IMFs. On the other hand, if the image set group 24 does not contain image information IFM corresponding to the determined latent variable Z, the generation unit 14 generates an image IL by inputting the latent variable Z to the decoder unit of the generative model 22 as described above. Then, the generation unit 14 sets the determined latent variable Z and the generated image IL as determined image information IMFs.

[0033] The generation unit 14 transmits to the control device 40 response information AI including the representative image information IFMs and the determined image information IMFs.

[0034] As shown in FIG. 6, the acquisition unit 43 of the control device 40 acquires the response information AI via the device interface 58 in step S40.

[0035] In step S50, the image display control unit 47 of the control device 40 displays the reference image ILs, which is the image IL of the representative image information IFMs included in the response information AI, and the determined image ILd, which is the image IL included in the determined image information IMFs, as an image group IG on the display device 80.

[0036] As shown in Fig. 8, the reference image ILs and the determined image ILd that make up the image group IG are displayed on the display device 80 so as to satisfy predetermined image display conditions. In this embodiment, the reference image ILs and the determined image ILd are displayed on the display device 80 in a 3 x 3 grid pattern with the reference image ILs at the center. To distinguish between the multiple determined images ILd, numbers 1 to 8 are added to the end of the images. The positional relationship of the multiple determined images ILd1 to ILd8 with respect to the reference image ILs displayed on the display device 80 may be correlated with their positional relationship in a predetermined space.

[0037] As described above, the image display control unit 47 causes the display device 80 to display the first image group IG1 as an image group IG including the first reference image ILs as a reference image ILs associated with the first class as the image class CL selected by the class selection image IM1, and the plurality of first determined images ILd1-ILd8 determined by the server 10 in accordance with the first reference image ILs. Taking FIG. 8 as an example, the first class is the image class CL representing "dog," and the first reference image ILs is the image displayed at the center of the image group IG1 shown in FIG. 8. Furthermore, as described above, the image display control unit 47 causes the display device 80 to display, as the image group IG, the plurality of images ILs, ILd1-ILd8 located within a search region SR centered on the reference image ILs in a predetermined space. Furthermore, as described above, the image display control unit 47 causes the display device 80 to display, as the image group IG, a predetermined number N of images ILs, ILd1-ILd8 from the plurality of images located within the search region SR.

[0038] Although the reference image ILs is determined by selecting one image information IFM as representative image information IFMs from the image information IFM extracted by the server 10, this is not limiting. For example, the user may specify the reference image ILs. In this case, reference specification information for identifying the reference image ILs specified by the user, such as an image identifier ID, is included in the request information RI and transmitted to the server 10. For example, as shown in FIG. 8, one image is selected from the first image group IG1 displayed on the display device 80 using a cursor or the like. In FIG. 8, it is assumed that the first determined image ILd6 is selected by the user. When the user selects an image to be the reference image IL, selection information SI indicating the selection may be generated by the display control unit 44 and displayed on the display device 80. The selection information SI is, for example, a dashed frame image surrounding the selected reference image. A case in which the first determined image ILd6 is selected as the reference image by the user, and the point image IM2b shown in FIG. 8 is moved to the right to expand the search area SR, and then the received image IM4 is selected will be described with reference to FIGS. 10 and 11. That is, the following describes a case where the conditions for displaying the image group IG via the condition input image IM have changed since the previous time. When the conditions for displaying the image group IG have changed, the acquisition unit 43 generates new request information RI and transmits it to the server 10, as shown in step S30 in FIG.

[0039] Fig. 10 is a diagram illustrating a condition input image IM and an image group IG2 displayed on the display device 80. Fig. 11 is a diagram illustrating step S5 in Fig. 6, which is executed in response to newly generated request information RI. For ease of understanding, the request information RI generated last time will also be referred to as previous request information RI1, and the newly generated request information RI will also be referred to as current request information RI2. Furthermore, the response information AI to the previous request information RI1 will also be referred to as previous response information AI1, and the response information AI to the current request information RI2 will also be referred to as current response information AI2.

[0040] Upon receiving the current request information RI2, the generation unit 14 of the server 10 generates current response information AI2 in response to the current request information RI2 in step S35 shown in FIG. 6. The current request information RI2 includes reference designation information indicating that the determined image ILd6 shown in FIG. 8 is to be the reference image ILsa. Therefore, the generation unit 14 of the server 10 selects the determined image ILd6 corresponding to the reference designation information as the reference image ILs. Furthermore, as described above, the generation unit 14 determines a predetermined number N of latent variables Z from within the search area SRa centered on the coordinate point in the predetermined space between the reference image ILsa and the image class CL associated with the reference image ILsa so as to satisfy the determination condition. As shown in FIG. 11, the search area SRa for the current request information RI2 is larger than the search area SR for the previous request information RI1. Therefore, image information IFMs for image classes CL of "fish" and "cat" as well as "dog" are located within the search area SR. The generation unit 14 sets image information IFM corresponding to the determined latent variables Z as determined image information IMFs. The generation unit 14 then sets the determined latent variable Z and the generated image IL as determined image information IMFs. The generation unit 14 then transmits current response information AI2, which includes representative image information IFMs including the reference image ILsa and the determined determined image information IMFs, to the control device 40.

[0041] 6, when the control device 40 acquires current response information AI2, in step S50, the control device 40 uses the current response information AI2 to display a second image group IG2 as the image group IG on the display device 80. Specifically, as shown in FIG. 10, the image display control unit 47 displays the reference image ILsa included in the second image group IG2 and the determined images ILd1a to ILd8a as the images IL included in the determined image information IMFs on the display device 80.

[0042] As described above, the image display control unit 47 selects one of the images ILs, ILd1 to ILd8 of the first image group IG1 shown in Fig. 8, and sets the selected image as a new reference image ILsa. The image display control unit 47 then causes the display device 80 to display the second image group IG2, which includes the new reference image ILsa and a plurality of second determined images ILd1a to ILd8a as a plurality of determined images determined in accordance with the new reference image ILsa.

[0043] 10 also includes images ILd1a, ILd4a, and ILd6a of "fish" and "cat," which are of an image class CL different from the selected image class CL of "dog," due to the widening of the search area SR. In other words, when there are other-class images associated with other image classes different from the image class of the reference image ILsa within the search area SRa, the image display control unit 47 causes the display device 80 to display the image group IG2 including the other-class images.

[0044] Next, a case where the image class CL selected via the condition input image IM is changed will be described with reference to Figures 12 and 13. Figure 12 is a diagram illustrating the condition input image IM and the image group IG3 displayed on the display device 80. Figure 13 is a diagram illustrating step S5 in Figure 6, which is executed in response to newly generated request information RI. For ease of understanding, the previously generated request information RI will also be referred to as previous request information RI1, and the newly generated request information RI will also be referred to as current request information RI3. Furthermore, the response information AI to the previously generated request information RI1 will also be referred to as previous response information AI1, and the response information AI to the current request information RI3 will also be referred to as current response information AI3.

[0045] For example, suppose the user changes the conditions shown in the condition input image IM in Fig. 8 to the conditions shown in the condition input image IM in Fig. 12 and selects the reception image IM4. Specifically, the user selects "fish" instead of "dog" for the image class CL via the class selection image IM1. In this case, in step S30 shown in Fig. 6, the acquisition unit 43 generates current request information RI3 including the specified image class CLs and the specified search region SRs. The acquisition unit 43 transmits the generated current request information RI3 to the server 10.

[0046] Upon receiving the current request information RI3, the server 10 executes step S35 shown in FIG. 6. Since step S35 has already been described, details of step S35 are omitted. In step S35, the generation unit 14 of the server processor 12 selects a reference image ILsb. As shown in FIG. 13, the designated image class CLs included in the current request information RI3 is “fish.” Therefore, the generation unit 14 selects one representative image information IFMs from the image information IFMs associated with “fish” as the image class CL. The generation unit 14 then selects the image IL included in the representative image information IFMs as the reference image ILsb. Furthermore, as described above, the generation unit 14 extracts determined image information IMFs by determining a predetermined number N of image information IFMs from the image information IFMs corresponding to the latent variable Z located within a search region SRb centered on the representative image information IFMs in a specified space. The generation unit 14 then transmits current response information AI3 including the representative image information IFMs and the determined determined image information IMFs to the control device 40.

[0047] As shown in FIG. 6, the acquisition unit 43 of the control device 40 acquires current response information AI3 via the device interface 58 in step S40. In step S50, the image display control unit 47 of the control device 40 displays, as an image group IG3, on the display device 80, a reference image ILsb, which is the image IL of the representative image information IFMs included in the current response information AI3, and determined images ILd1b to ILd8b, which are images IL included in the determined image information IMFs. As shown in FIG. 12, the image display control unit 47 displays, as an image group IG3, nine images ILsb, ILd1b to ILd8b, whose image class CL is "fish." The reference image ILsb is also referred to as a second reference image ILsb, and the determined images ILd1b to ILd8b are also referred to as second determined images ILd1b to ILd8b.

[0048] As described above, when the second class as the image class CL is selected via the class selection image IM1 instead of the first class as the image class CL, the image display control unit 47 executes the following: That is, the image display control unit 47 causes the display device 80 to display the second image group IG3 including the second reference image ILsb as a reference image associated with the selected second class, and a plurality of second determined images ILd1b to ILd8b determined in accordance with the second reference image ILsb.

[0049] As described with reference to FIGS. 9, 11, and 13, the plurality of decided images ILd1 to ILd8, ILd1a to ILd8a, and ILd1b to ILd8b are located within a search region SR designated in a predetermined space.

[0050] 14 is a diagram illustrating a mapping image IMp displayed on the display device 80. When the mapping selection image IM3 is designated and the received image IM4 is selected, information for acquiring information necessary for generating the mapping image IMp is included in request information RI and transmitted to the server 10. The control device 40 causes the mapping image IMp to be displayed on the display device 80 instead of or in addition to the image group IG. Note that the mapping image IMp may be generated by the control device 40.

[0051] As shown in FIG. 14 , the mapping image IMp is an image in which image information IFM is visually arranged in a predetermined space. In this embodiment, the mapping image IMp includes coordinate axes of the predetermined space and images IL arranged in the predetermined space. Note that the mapping image IMp may also include a class image representing an image class CL for each region in which the images IL are located. The mapping display control unit 49 may display all images IL included in the mapping information Mp as the mapping image IMp on the display device 80, or may display a portion of the images IL included in the mapping information Mp as the mapping image IMp on the display device 80. The number of images IL constituting the mapping image IMp may be specified by the user. The mapping display control unit 49 acquires the mapping information Mp from the server 10 as part of the response information AI or separately from the response information AI and stores it in the device storage unit 50. That is, the device storage unit 50 stores the mapping information Mp. The mapping display control unit 49 generates the mapping image IMp using the mapping information Mp stored in the device storage unit 50 and displays it on the display device 80. The mapping image IMp may also include an image showing the current position of the reference image ILs and the range of the search region SR.

[0052] As described above, the mapping display control unit 49 generates a mapping image IMp in which at least some of the images IL included in the mapping information Mp are arranged in a predetermined space, and causes the display device 80 to display the generated mapping image IMp.

[0053] According to the first embodiment, as shown in FIG. 8, a user can select an image class CL of an image to be searched for using a class selection image IM1. This allows the user to efficiently search for a desired image, such as an image of a dog or an image of a fish. Furthermore, according to the first embodiment, as shown in FIG. 8, a user can specify the size of a search region SR in a predetermined space using a region selection image IM2. This allows the user to specify the search region SR of an image in a predetermined space in addition to selecting an image class CL, thereby enabling more efficient search for a desired image. For example, by reducing the search region SR, the user can display images that have the same image class CL as the reference image ILs and are more similar to the reference image ILs as the image group IG on the display device 80. Furthermore, for example, by increasing the search region SR, the user can display images that are less similar to the reference image ILs as the image group IG on the display device 80.

[0054] 10 and 11, when other-class images ILd1a, ILd4a, ILd6a associated with an image class CL different from that of the reference image ILs are located within the search region SRa, the other-class images ILd1a, ILd4a, ILd6a can also be displayed on the display device 80. This allows the user to easily search for a desired image even when at least one of a plurality of desired images is the other-class image ILd1a, ILd4a, ILd6a associated with an image class CL different from that of the reference image ILs.

[0055] Furthermore, according to the first embodiment, the control device 40 acquires and stores the mapping information Mp shown in FIG. 4 from the server 10, and can display the mapping image IMp corresponding to the mapping information Mp on the display device 80. This allows the user to visually grasp the position of the image IL in a predetermined space. Therefore, the user can search for the desired image IL with reference to the mapping image IMp. For example, the user can determine the size of the search area SR with reference to the mapping image IMp.

[0056] Furthermore, according to the first embodiment, in addition to the image IL, the image class CL is also considered as a factor of similarity, so that images IL of the same image class CL can be arranged in a relatively close area in a predetermined space. This allows the user to search for a desired image more efficiently. For example, in a predetermined space, images IL of the same image class CL as the reference image ILs are arranged in a position close to the reference image ILs. Therefore, by having the control device 40 search for images in a search area SR centered on the reference image ILs, the images IL of the same image class CL as the reference image ILs can be displayed on the display device 80. This allows the user to easily search for images IL associated with a desired image class CL.

[0057] 8 and 9, the control device 40 displays, on the display device, a plurality of images ILs, ILd1-ILd8 located within a search region SR centered on the reference image ILs, allowing the user to easily search for a desired image from among the plurality of images ILs, ILd1-ILd8. According to the first embodiment, the control device 40 displays, on the display device 80, a predetermined number N of images ILs, ILd1-ILd8 located within a search region SR centered on the reference image ILs, allowing the user to easily search for a desired image from among the predetermined number N of images ILs, ILd1-ILd8. Note that this predetermined number N can be changed by the user.

[0058] Furthermore, according to the first embodiment, as shown in FIG. 14, the mapping image IMp displayed on the display device 80 allows the user to easily grasp the position of each image IL in a predetermined space.

[0059] B. Second embodiment: 15 is a diagram showing a condition input image IMt in the second embodiment. The second embodiment differs from the first embodiment in the items of the condition input image IMt that the control device 40 causes the display device 80 to display. The other configurations in the second embodiment are the same as those in the first embodiment, so the same reference numerals are used for the same configurations and the description thereof will be omitted as appropriate.

[0060] As in the first embodiment, in the second embodiment, the control device 40 generates a condition input image IMt and displays it on the display device 80. The condition input image IMt includes a class selection image IM1, an area selection image IM2, a mapping selection image IM3, and a reception image IM4, as well as a history selection image IM6 and a designation reception image IM7. For example, in any of the processes in step S10 shown in FIG. 6 , the history selection image IM6 and the designation reception image IM7 are generated by the display control unit 44 and displayed on the display device 80. In this embodiment, the class selection display control unit 46 displays the generated history selection image IM6 and designation reception image IM7 on the display device 80.

[0061] The history selection image IM6 is an image for selecting a recommended image ILR generated using the history of an image group IG previously displayed on the display device 80 as a reference image ILs. The recommended image ILR is an image previously displayed on the display device 80 as a reference image ILs. The recommended image ILR is obtained, for example, by the control device 40 from the server 10. In FIG. 15 , there is one recommended image ILR, but there may be multiple recommended images ILR if there are multiple reference images ILs previously displayed. The recommended image ILR is an image associated with the image class CL specified in the class selection image IM1. The user selects the recommended image ILR of the history selection image IM6 with a cursor or the like, and also selects the received image IM4. As a result, request information RI including information specifying the selected recommended image ILR as the reference image ILs is generated and transmitted to the server 10.

[0062] The designation acceptance image IM7 is an image that accepts designation of a distance and a direction from a predetermined position in a predetermined space. Similar to the mapping image IMp shown in FIG. 14, the designation acceptance image IM7 is an image in which image information IFM is visually arranged in a predetermined space. The designation acceptance image IM7 includes coordinate axes of the predetermined space and an image IL arranged in the predetermined space. The class selection display control unit 46 may display all images IL included in the mapping information Mp as the designation acceptance image IM7 on the display device 80, or may display only a portion of the images IL included in the mapping information Mp as the designation acceptance image IM7 on the display device 80. The user designates an image IL to be a reference image ILs from among the multiple images IL included in the designation acceptance image IM7 using a cursor or the like. The user also designates a distance and a direction from the designated image ILe on the designation acceptance image IM7 using a cursor or the like. For example, the user designates the designated image ILe by drawing a frame-shaped line Re on the designation acceptance image IM7 so as to surround the designated image ILe. The designated image IL included in the image group IG is designated from within the area surrounded by the frame-shaped line Re. That is, the user uses the frame-shaped line Re to specify the distance and direction from the coordinate point of the designated image ILe, which is a predetermined position. When the reception image IM4 is selected, the designated image ILe, distance, and direction specified in the designation reception image IM7 are included as information in the request information RI and transmitted to the server 10. Note that the image class CL associated with the reference image ILs specified by the designation reception image IM7 may be automatically input into the class selection image IM1. Furthermore, the point image IM2b of the area selection image IM2 may automatically slide according to the distance and direction specified by the designation reception image IM7, i.e., according to the area enclosed by the frame-shaped line Re.

[0063] The second embodiment provides the same effects as the first embodiment in that it has the same configuration. For example, in the second embodiment, as shown in FIG. 15 , a user can select an image class CL of an image to be searched for using a class selection image IM1. This allows the user to efficiently search for a desired image, such as an image of a dog or an image of a fish. Furthermore, according to the second embodiment, the user can efficiently search for a desired image by specifying a distance and direction in a predetermined space via a designation acceptance image IM7 or by using a history selection image IM6 that uses a history of an image group IG previously displayed on the display device 80. Note that in the second embodiment, the class selection display control unit 46 may display at least one of the designation acceptance image IM7 and the history selection image IM6 on the display device 80.

[0064] C. Other Embodiments: C1. Alternative Embodiment 1: In the above embodiments, the image class CL is used to classify animal types, but the present invention is not limited to this and may be used to classify data features. For example, the image class CL may be used to classify the "hair color" or "face orientation" of human face images.

[0065] C2. Alternative Embodiment 2: The display system 5 in each of the above embodiments can be used for various purposes. For example, it may be used to search for a desired image, as in each of the above embodiments, or it may be used to collect a history of images IG displayed on the display device 80, generate a new image IL from the history, and display it on the display device 80.

[0066] C3. Alternative Embodiment 3: In each of the above embodiments, the image IL is a two-dimensional image, but it may be a stereoscopic image or a moving image.

[0067] D. Other forms: The present disclosure is not limited to the above-described embodiments and can be realized in various forms without departing from the spirit thereof. For example, the present disclosure can also be realized in the following aspects. The technical features in the above embodiments corresponding to the technical features in each aspect described below can be appropriately replaced or combined to solve some or all of the problems of the present disclosure or to achieve some or all of the effects of the present disclosure. Furthermore, if a technical feature is not described as essential in this specification, it can be appropriately deleted.

[0068] (1) According to a first aspect of the present disclosure, there is provided a control device for controlling image display, the control device including: a class selection display control unit that causes a display device to display a class selection image that accepts selection of an image class to be specified from among a plurality of image classes; and an image display control unit that causes the display device to display a first image group as a group of images including a first reference image as a reference image associated with a first class as the selected image class and a plurality of first determined images as a plurality of determined images determined according to the first reference image, wherein the image display control unit causes the display device to display a second image group as the group of images including the selected image as a new reference image and a plurality of second determined images as the plurality of determined images determined according to the new reference image when one of the images in the first image group is selected; and when a second class as the image class is selected instead of the first class via the class selection image, the image display control unit causes the display device to display a second image group including the second reference image as the reference image associated with the selected second class and a plurality of second determined images determined according to the second reference image. According to this aspect, the user can select an image class using the class selection image, and thus can efficiently search for a desired image.

[0069] (2) In the above aspect, the display device may further include an area display control unit that causes the display device to display an area selection image that specifies the size of a search area in a predetermined space that defines image information including the image, and the plurality of determined images may be positioned within the specified search area in the predetermined space. According to this aspect, the user can specify the search area for images in the predetermined space in addition to selecting an image class, thereby enabling more efficient searching for a desired image.

[0070] (3) In the above aspect, when an other-class image associated with an other-class image class different from the image class of the reference image is present in the search area, the image display control unit may cause the display device to display the image group including the other-class image. According to this aspect, when an other-class image is located in the search area, the other-class image can also be displayed on the display device.

[0071] (4) In the above aspect, the image information may include the image and an image class associated with the image, and the device may further include a device storage unit that stores mapping information in which the plurality of pieces of image information are represented by the number of dimensions of the predetermined space and are mapped onto the predetermined space. According to this aspect, since the control device has the mapping information, it can display a mapping image corresponding to the mapping information on the display device, allowing the user to visually grasp the position of the image in the predetermined space.

[0072] (5) In the above embodiment, the plurality of image information may be mapped onto the predetermined space according to the similarity between the unit sets, with the image and the image class being treated as one unit set. According to this embodiment, since the image class as well as the image are considered as elements of similarity, images of the same image class can be arranged in a relatively close area in the predetermined space. This allows the user to search for a desired image more efficiently.

[0073] (6) In the above aspect, the image display control unit may cause the display device to display, as the image group, a plurality of the images located within the search area centered on the reference image in the predetermined space. According to this aspect, by causing the display device to display a plurality of images located within the search area centered on the reference image, the user can easily search for a desired image from among the plurality of images.

[0074] (7) In the above aspect, the image display control unit may cause the display device to display a predetermined number of the images from the plurality of images located within the search area as the image group. According to this aspect, by causing the display device to display a predetermined number of images located within a search area centered on a reference image, a user can easily search for a desired image from among the predetermined number of images.

[0075] (8) In the above embodiment, the class selection display control unit may further cause the display device to display at least one of a designation acceptance image that accepts designation of a distance and a direction from a predetermined position in the specified space, and a history selection image for selecting a recommended image generated using a history of the group of images previously displayed on the display device as the reference image. According to this embodiment, the user can efficiently search for a desired image by designating a distance and a direction in the specified space via the designation acceptance image or by using a history selection image that uses a history of the group of images previously displayed on the display device.

[0076] (9) In the above aspect, a mapping display control unit may further be provided that causes the display device to display a mapping image in which at least some of the images included in the mapping information are arranged in the predetermined space. According to this aspect, the mapping image allows a user to easily grasp the positions of the images in the predetermined space.

[0077] (10) According to a second aspect of the present disclosure, there is provided a computer program for controlling display of an image. This computer program causes a computer to execute the following functions: (a) displaying on a display device a class selection image that accepts selection of an image class to be specified from among a plurality of image classes; and (b) displaying on the display device a first image group as an image group having a first reference image as a reference image associated with a first class as the selected image class, and a plurality of first determined images as a plurality of determined images determined in accordance with the first reference image. The function (b) has the function of, when one of the images in the first image group is selected, displaying on the display device a second image group that has the selected image as a new reference image and a plurality of second determined images as the plurality of determined images determined in accordance with the new reference image; and, when a second class as the image class is selected in place of the first class via the class selection image, the function (b) has the function of displaying on the display device a second image group that has the second reference image as the reference image associated with the selected second class, and a plurality of second determined images determined in accordance with the second reference image. According to this aspect, the user can select an image class using the class selection image, and thus can efficiently search for a desired image.

[0078] The present disclosure may be realized in various forms other than those described above, such as a method for controlling image display, a display system, or a non-transitory storage medium having a computer program recorded thereon. [Explanation of symbols]

[0079] 5...Display system, 10...Server, 12...Server processor, 14...Generation unit, 20...Server memory unit, 22...Generative model, 24...Image set group, 28...Server interface, 40...Control device, 42...Device processor, 43...Acquisition unit, 44...Display control unit, 46...Class selection display control unit, 47...Image display control unit, 48...Area display control unit, 49...Mapping display control unit, 50...Device memory unit, 58...Device interface, 80...Display device, 99...Input device, AI...Response information, CL...Image class, CLs...Specified image class, ID...Image identifier, IFM...Image information, IFMs...Representative image Image information, IG, IG1, IG2, IG3...image group, IL...specified image, ILs, ILsa, ILsb...reference image, IM, Imt...condition input image, IM1...class selection image, IM2...area selection image, IM2a...bar image, IM2b...point image, IM3...mapping selection image, IM4...reception image, IM6...history selection image, IM7...specified reception image, IMF...image information, IMFs...decision image information, IMp...mapping image, INT...internet, LT...location information, Mp...mapping information, RI...request information, Re...line, SI...selection information, SR...search area, SRs...specified search area, Z...latent variable

Claims

1. A control device for controlling display of an image, a class selection display control unit that displays, on a display device, a class selection image that accepts selection of the image class to be specified from among a plurality of image classes; an image display control unit that causes the display device to display a first image group as an image group including a first reference image as a reference image associated with a first class as the selected image class, and a plurality of first determined images as a plurality of determined images determined in accordance with the first reference image, the image display control unit, when one of the images in the first image group is selected, causes the display device to display a second image group as the image group having the selected image as a new reference image and a plurality of second determined images as the plurality of determined images determined in accordance with the new reference image; When a second class is selected as the image class instead of the first class via the class selection image, the image display control unit causes the display device to display a second image group having a second reference image as the reference image associated with the selected second class, and a plurality of second determined images determined according to the second reference image.

2. The control device according to claim 1, further comprising: an area display control unit that causes the display device to display an area selection image that specifies the size of a search area in a predetermined space that defines image information including the image; A control device, wherein the plurality of decision images are located within the specified search area in the predetermined space.

3. The control device according to claim 2, The image display control unit is a control device that, if there is an other-class image in the search area that is associated with an other image class different from the image class of the reference image, causes the display device to display the group of images including the other-class image.

4. The control device according to claim 2, the image information includes the image and an image class associated with the image; The control device further comprises a device storage unit that stores mapping information in which the plurality of pieces of image information are represented by the number of dimensions of the predetermined space and are mapped onto the predetermined space.

5. The control device according to claim 4, The control device wherein the plurality of pieces of image information are mapped onto the predetermined space according to the similarity between the unit sets, with the image and the image class being treated as one unit set.

6. The control device according to any one of claims 2 to 5, The image display control unit causes the display device to display, as the image group, a plurality of the images located within the search area centered on the reference image in the predetermined space.

7. 7. The control device according to claim 6, The image display control unit causes the display device to display a predetermined number of the images from the plurality of images located within the search area as the image group.

8. The control device according to claim 6, The class selection display control unit further causes the display device to display at least one of a designation acceptance image that accepts designation of the distance and direction from a predetermined position in the specified space, and a history selection image that selects a recommended image generated using the history of the group of images that have been displayed on the display device in the past as the reference image.

9. The control device according to claim 4, further comprising: a mapping display control unit that causes the display device to display a mapping image in which at least some of the images included in the mapping information are arranged in the predetermined space.

10. 1. A computer program for controlling the display of images, comprising: (a) a function of displaying on a display device a class selection image for accepting selection of an image class to be specified from among a plurality of image classes; (b) causing the computer to execute a function of displaying, on the display device, a first image group as an image group including a first reference image as a reference image associated with a first class as the selected image class, and a plurality of first determined images as a plurality of determined images determined in accordance with the first reference image; The function (b) is a function of selecting one of the images in the first image group, and displaying on the display device a second image group including the selected image as a new reference image and a plurality of second determined images as the plurality of determined images determined in accordance with the new reference image; When a second class is selected as the image class in place of the first class via the class selection image, the function (b) includes a function for displaying on the display device a second group of images including a second reference image as the reference image associated with the selected second class and a plurality of second determined images determined in accordance with the second reference image.

Images