Information processing apparatus, information processing method, and information processing program

Abstract

To easily generate a realistic image on the basis of an image in which an imaginary landscape is drawn.SOLUTION: The information processing apparatus 100 according to the present embodiment includes an estimation unit 121, an adjustment unit 122, and a generation unit 123. The estimation unit 121 estimates a prompt sentence for generating an image by the generation model for the aerial scenic image. An adjustment part 122 adjusts the prompt sentence estimated by the estimation part 121 so that an image generated by the prompt sentence becomes a realistic image. The generation unit 123 generates an image by the generation model using the prompt sentence adjusted by the adjustment unit 122.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and an information processing program. [Background technology]

[0002] Conventionally, an image generation model has been used that generates an image that reflects text specified by a user (see, for example, Patent Document 1). The image generation model receives a prompt statement that specifies the content of the image to be generated, and generates an image that reflects the received content, so the user is required to understand the prompt that will generate the image that he or she desires. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2024-060907 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the conventional technology, it is not easy to generate a realistic image based on an image depicting a fictional landscape. For example, in the conventional technology, a user is required to set a prompt, so it is difficult to appropriately set the prompt for generating an image depicting a fictional landscape image as a realistic landscape, and it is not easy to convert the fictional landscape image into a realistic image. [Means for solving the problem]

[0005] In order to solve the above-mentioned problems and achieve the objectives, the information processing device of the present invention is characterized by having an estimation unit that estimates a prompt sentence for generating an image using a generative model for a fictional landscape image, an adjustment unit that adjusts the prompt sentence estimated by the estimation unit so that the image generated using the prompt sentence becomes a realistic image, and a generation unit that generates an image using the generative model using the prompt sentence adjusted by the adjustment unit. [Effects of the Invention]

[0006] According to the present invention, it is possible to easily generate a realistic image based on an image depicting a fictional landscape. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram illustrating an example of the overall configuration of a system according to the first embodiment. [Figure 2] FIG. 2 is a block diagram illustrating an example of the configuration of the information processing device according to the first embodiment. [Figure 3] FIG. 3 is a diagram illustrating an example of data registered in the storage unit according to the first embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of a method for adjusting a prompt sentence according to the first embodiment. [Figure 5] FIG. 5 is a diagram illustrating a specific example of the processing performed by the estimation unit and the adjustment unit according to the first embodiment. [Figure 6] FIG. 6 is a diagram illustrating a specific example of the processing of the generating unit and the identifying unit according to the first embodiment. [Figure 7] FIG. 7 is a flowchart showing an example of the flow of processing by the information processing device according to the first embodiment. [Figure 8] FIG. 8 is a diagram illustrating an example of the overall configuration of a system according to the second embodiment. [Figure 9] FIG. 9 is a diagram illustrating an outline of the processing performed by the travel suggestion device according to the second embodiment. [Figure 10] FIG. 10 is a block diagram illustrating an example of the configuration of a travel suggestion device according to the second embodiment. [Figure 11] FIG. 11 is a diagram illustrating an example of processing by an image posting unit according to the second embodiment. [Figure 12] FIG. 12 is a diagram illustrating a specific example of the processing performed by the image posting unit and the image receiving unit according to the second embodiment. [Figure 13] FIG. 13 is a diagram illustrating a specific example of the processing performed by the place name identification unit and the travel plan presenting unit according to the second embodiment. [Figure 14] FIG. 14 is a diagram illustrating a specific example of the processing performed by the image posting unit and the reward granting unit according to the second embodiment. [Figure 15] FIG. 15 is a flowchart illustrating an example of the processing flow of the travel suggestion device according to the second embodiment. [Figure 16] FIG. 16 is a diagram illustrating an example of a computer that executes a program according to the first or second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments of an information processing device, an information processing method, and an information processing program according to the present application will be described in detail with reference to the accompanying drawings. Note that the information processing device, the information processing method, and the information processing program according to the present application are not limited to these embodiments.

[0009] [Embodiment 1] (Overall composition) First, an overview of the information processing system according to the present embodiment 1 will be described. Fig. 1 is a diagram showing an example of the overall configuration of the system according to the present embodiment 1. The information processing system shown in Fig. 1 includes an information processing device 100 that generates an image that depicts a received fictional landscape image as a realistic landscape, a place name search server 200 that stores a plurality of sets of real landscape images and information on the locations where the images were taken, and a user terminal 300 that transmits to the information processing device 100 a base image that is used to identify places where similar landscapes can be seen.

[0010] The information processing device 100 is an information processing device that generates an image that depicts a fictional landscape image as a realistic landscape based on the fictional landscape image received from the user terminal 300, and is realized by, for example, a computer, a cloud system, etc. For example, the information processing device 100 generates an image that depicts a realistic building based on a CG (Computer Graphics) image that depicts a building in a game received from the user terminal 300.

[0011] Furthermore, the information processing device 100 can use the generated realistic image to identify the place name of a place where a landscape similar to that image can be seen. For example, by using the realistic landscape image generated based on the above-mentioned CG image, the information processing device 100 can identify the place name of a real place where a building similar to the building depicted in the landscape image can be seen.

[0012] Place name search server 200 is an information processing device that stores a set of photographed images of real-world scenery and information on the location where the images were taken, and is realized by a server device, a cloud system, etc. For example, place name search server 200 stores multiple sets of information on the images and their photographing locations described above, and outputs information on an extracted set of images and their photographing locations in response to a search instruction from information processing device 100.

[0013] The user terminal 300 is an information processing terminal that transmits images other than real landscapes, such as CG images depicting in-game buildings or illustrated images depicting fictional landscapes, to the information processing device 100, and is realized by a smartphone or tablet terminal. For example, the user terminal 300 transmits an illustrated image other than real landscapes to the information processing device 100, and after the information processing device 100 identifies a location, displays on the screen a real image of a landscape similar to the illustrated image and information about the location where the image was taken, as information about the place name of the identified location.

[0014] (Overview of processing by information processing device 100) Next, the processing details of the information processing device 100 according to the first embodiment will be described. The information processing device 100 estimates a prompt sentence for generating an image of a fictional landscape using a generative model, and adjusts the estimated prompt sentence so that the image generated by the prompt sentence becomes a realistic image. Then, the information processing device 100 generates an image using the generative model using the adjusted prompt sentence.

[0015] For example, the information processing device 100 performs a reverse prompt process for estimating a prompt sentence for generating an illustration image, which is received from the user terminal 300 and depicts a fictional landscape other than a real landscape, using a generative model.

[0016] The information processing device 100 then adjusts the prompt sentence estimated by the reverse prompt process, for example, by deleting phrases that specify the image depiction style, such as anime, comic, or illustration, so that a more realistic image is generated.The information processing device 100 then generates an image using a generative model using the prompt sentence adjusted to generate a more realistic image.

[0017] This allows the information processing device 100 to generate an image depicting a realistic fictional landscape from an image depicting a fictional landscape received from a user, and thus makes it possible to easily generate a realistic image based on an image depicting a fictional landscape.

[0018] (Configuration of information processing device 100) An example of the functional configuration of the information processing device 100 will be described below. Fig. 2 is a block diagram showing an example of the configuration of the information processing device according to embodiment 1. As shown in Fig. 2, the information processing device 100 has a communication unit 110, a control unit 120, and a storage unit 130.

[0019] The communication unit 110 is realized by, for example, a network interface card (NIC), etc. For example, the communication unit 110 controls communication regarding various information exchanged with connected devices, and mediates processing by the identification unit 124, which will be described later.

[0020] The storage unit 130 is realized by a storage device such as a RAM (Random Access Memory) or a hard disk, for example. The storage unit 130 stores data and programs necessary for various processes by the control unit 120. The storage device may be realized by a storage system or the like installed outside the information processing device 100. The storage unit 130 stores information such as image data of a fictional landscape image received from the user and text data of a prompt sentence by a reverse prompt.

[0021] Here, the data stored by the storage unit 130 at each processing stage described below will be described with reference to Fig. 3. Fig. 3 is a diagram showing an example of data registered in the storage unit according to the first embodiment. For example, the storage unit 130 stores registered data for the items "image number," "accepted image," "adjusted image," and "similar image."

[0022] The "image number" stores a number assigned to each image depicting a fictional landscape received from the user. The "received image" stores image data of the image depicting a fictional landscape received from the user and text data of the prompt sentence acquired when the estimation unit 121 performs reverse prompt processing on the image data, which will be described later.

[0023] The "adjusted image" stores text data of the prompt sentence adjusted by the adjustment unit 122 (described later) and image data generated by the generative model using the adjusted prompt sentence. The "similar image" stores image data of a photographed image of a real landscape identified by the identification unit 124 (described later) and information indicating the location where the image was taken.

[0024] As shown in FIG. 3, for example, for an image with image number "1," the storage unit 130 registers image data "a1.png" of an image depicting a fictional landscape received from the user, and a prompt text "a1.txt" obtained by performing reverse prompt processing on "a1.png."

[0025] The storage unit 130 also stores a prompt text "b1.txt" adjusted by the adjustment unit 122 (described later) and image data "b1.png" generated by the generation unit 123 (described later) using "b1.txt." The storage unit 130 also stores image data "c1.png" of a photographed image of a real landscape similar to "b1.png" by the identification unit 124 (described later), and "c1.txt" indicating the photographing location linked to "c1.png."

[0026] The control unit 120 is realized by a processor such as an integrated circuit, such as a CPU (Central Processing Unit), an MPU (Micro Processing Unit), an ASIC (Application Specific Integrated Circuit), or an FPGA (Field Programmable Gate Array), executing various programs stored in a storage device inside the information processing device 100 using a RAM or the like as a working area. In the example shown in FIG. 2 , the control unit 120 includes an estimation unit 121, an adjustment unit 122, a generation unit 123, and an identification unit 124.

[0027] The estimation unit 121 estimates a prompt sentence for generating a fictional landscape image using a generative model for the fictional landscape image. For example, the estimation unit 121 performs a reverse prompt process for estimating a prompt sentence for generating an illustration image, which is received from the user terminal 300 and depicts a fictional landscape other than a real landscape, using a generative model.

[0028] Here, the fictional landscape images accepted by the estimation unit 121 are not particularly limited as long as they depict some kind of fictional content, and include, for example, images depicting fictional landscapes within a game, landscape photos with characters taken using an in-game photo AI function, images depicting fictional dishes, images depicting fictional creatures, etc.

[0029] Furthermore, the format of the prompt sentence output by the estimation unit 121 is not particularly limited, and may be, for example, a prompt sentence in a format in which multiple words are listed, or a prompt sentence in a format in which multiple sentences of a certain length are composed.

[0030] The adjustment unit 122 adjusts the prompt sentence estimated by the estimation unit 121 so that the image generated by the prompt sentence becomes a realistic image. For example, the adjustment unit 122 deletes, from the prompt sentence estimated by the above-described estimation unit 121 through the reverse prompt processing, a phrase that specifies the nature of the image generated from the prompt sentence.

[0031] Specifically, the adjustment unit 122 adjusts the prompt sentence so that the generated image looks realistic by deleting, for example, a phrase specifying an expression technique such as "illustration," "anime screencap," or "pastel painting" that would make the landscape depicted in the image generated by the prompt sentence look unrealistic. Alternatively, the adjustment unit 122 can adjust the prompt sentence so that the generated image looks realistic by adding a phrase such as "realistic," instead of deleting the aforementioned phrase.

[0032] Here, the processing of the adjustment unit 122 will be described with a specific example with reference to Fig. 4. Fig. 4 is a diagram showing an example of a method for adjusting a prompt sentence according to the first embodiment. Unlike the above-described method for adjusting a prompt sentence, Fig. 4 shows an example in which a realistic image is generated by adjusting a parameter value, such as the noise removal strength, that determines how much an image generated by an adjusted prompt sentence should deviate from the received image.

[0033] 4, first, the estimation unit 121 performs a reverse prompt process on an image of a fictional building in the game received from the user, and obtains a prompt sentence represented by the sentence "A castle in the clouds...". Then, the adjustment unit 122 adds a description to the estimated prompt sentence, which adjusts a parameter value indicating how much the estimated prompt sentence should deviate from the received image represented by "Step: 20, Sampler: DPM++ 2M Karras...", thereby adjusting the prompt sentence to generate a more realistic image based on the received image.

[0034] This allows the adjustment unit 122 to adjust the prompt sentence to generate a realistic image by adding a description for setting a parameter value from the prompt sentence estimated for the image depicting a fictional landscape received from the user.

[0035] The generation unit 123 generates an image from a generative model using the prompt sentence adjusted by the adjustment unit 122. For example, the generation unit 123 sets, in the generative model, a prompt sentence whose wording has been adjusted by the adjustment unit 122 so that the generated image looks realistic, and acquires the output image.

[0036] The identification unit 124 identifies the shooting location of an image of a real landscape similar to the image generated by the generation unit 123. For example, the identification unit 124 searches the image generated by the generation unit 123 in a place name search server 200 that stores a set of an image of a real landscape and the shooting location of the image of the real landscape, thereby identifying the shooting location of the image of a real landscape similar to the generated image.

[0037] Specifically, the identification unit 124 searches for similar images to the realistic image generated by the generation unit 123 described above, for example, in the place name search server 200. Then, the identification unit 124 identifies the shooting location of an image of a real landscape similar to the generated image by referring to the place name indicating the shooting location of the image linked to the output similar image and the position information of the shooting location.

[0038] Furthermore, the identification unit 124 identifies the shooting location from, for example, a prompt sentence estimated for an image of a real landscape similar to the image generated by the generation unit 123. Specifically, the identification unit 124 searches the Internet for realistic images generated by the generation unit 123 and acquires the image of the real landscape. Thereafter, the identification unit 124 performs a reverse prompt process on the acquired image to extract characters indicating a place name from the estimated prompt sentence and identify the character as the shooting location of the acquired image.

[0039] (Example) Next, the overall processing flow of the information processing device 100 according to the first embodiment will be described with reference to Fig. 5 and Fig. 6. Fig. 5 is a diagram showing a specific example of the processing of the estimation unit and the adjustment unit according to the first embodiment, and Fig. 6 is a diagram showing a specific example of the processing of the generation unit and the identification unit according to the first embodiment. The examples of Fig. 5 and Fig. 6 show an example in which, based on a CG image depicting a fictional landscape in a game received from a user, a place name of a real place where a landscape similar to the CG image can be seen is identified.

[0040] First, as shown in Fig. 5, the information processing device 100 receives image data of a CG image depicting a fictional landscape in a game from the user terminal 300. Then, the information processing device 100 performs reverse prompt processing on the received CG image and infers a prompt sentence such as "many mountains, river, cloudy..., CG" (Fig. 5(1)).

[0041] Next, the information processing device 100 adjusts the prompt sentence so that a realistic image is generated by deleting the word "CG," which specifies that the image to be generated will be CG-style, from the estimated prompt sentence (Figure 5(2)).

[0042] Next, as shown in FIG. 6, the information processing device 100 sets the prompt sentence, which has been adjusted to generate a realistic image, in the generation model, and generates a fictional landscape image depicting a realistic landscape (FIG. 6(3)).

[0043] The information processing device 100 then searches the place name search server 200 for the fictional landscape image depicting the generated realistic landscape image, identifies a landscape photograph similar to the fictional landscape image, and identifies the shooting location, "XX State, USA," linked to the image data of the identified landscape photograph (Figure 6(4)).

[0044] Through the series of processes described above, the information processing device 100 can identify the shooting location of a landscape similar to the landscape depicted in a CG image of a fictional landscape received from a user, and can therefore identify the place name of a real place where a landscape similar to the fictional landscape can be seen from the image depicting the fictional landscape.

[0045] (flowchart) Next, an example of processing by the information processing device 100 according to the first embodiment will be described with reference to Fig. 7. Fig. 7 is a flowchart showing an example of the flow of processing by the information processing device according to the first embodiment. Note that the steps in the flowchart shown in Fig. 7 may be executed in a different order, and some processing may be added or omitted.

[0046] First, the estimation unit 121 executes a reverse prompt to estimate a prompt sentence for generating a fictional landscape image (S101). Then, the adjustment unit 122 adjusts the estimated prompt sentence so that a realistic image is generated (S102).

[0047] Next, the generation unit 123 generates an image from the generative model using the adjusted prompt sentence (S103). Then, the identification unit 124 searches for the generated image in the place name search server 200, which stores multiple photographed images of real scenery (S104). Thereafter, the identification unit 124 identifies the photographing location associated with the retrieved image of real scenery (S105), and the information processing device 100 ends the process.

[0048] (effect) The information processing device 100 according to the first embodiment includes an estimation unit 121, an adjustment unit 122, and a generation unit 123. The estimation unit 121 estimates a prompt sentence for generating an image of a fictional landscape using a generative model. The adjustment unit 122 adjusts the prompt sentence estimated by the estimation unit 121 so that the image generated by the prompt sentence becomes a realistic image. The generation unit 123 generates an image using the generative model, using the prompt sentence adjusted by the adjustment unit 122.

[0049] This allows the information processing device 100 to generate an image depicting a realistic fictional landscape from an image depicting a fictional landscape received from a user. As a result, an image of a fictional landscape depicted in an illustrated style, graffiti, or the like can be easily converted into a realistic image without the user having to set a prompt indicating the characteristics of the image of the fictional landscape.

[0050] The information processing device 100 also includes an identification unit 124. The identification unit 124 identifies the location where an image of a real landscape similar to the image generated by the generation unit 123 was captured. In this way, the identification unit 124 identifies the place name of a place where a landscape similar to the generated realistic image can be seen, and can therefore identify the place name of a real place where a similar landscape can be seen from the fictional landscape image.

[0051] Furthermore, the adjustment unit 122 deletes, from the estimated prompt sentence, a phrase that specifies the nature of an image to be generated from the prompt sentence. As a result, the adjustment unit 122 can delete a phrase that specifies an unrealistic illustration or painting from the prompt sentence estimated by the reverse prompt, and adjust the prompt sentence to one that generates a realistic image.

[0052] Furthermore, the identification unit 124 searches for the image generated by the generation unit 123 on a predetermined terminal that stores a set of an image of a real landscape and the location where the image of the real landscape was taken. This allows the identification unit 124 to appropriately search for an image similar to the realistic image generated by the generative model and acquire information on the location where the searched image was taken.

[0053] Furthermore, the identification unit 124 identifies the shooting location from the prompt sentence estimated for an image of a real landscape similar to the image generated by the generation unit 123. This allows the information processing device 100 to appropriately identify the shooting location of an image from a real landscape image obtained by searching for the generated realistic image.

[0054] [Embodiment 2] In the first embodiment, the process of identifying a location where a landscape similar to a fictitious landscape image received from a user can be seen has been described, but in addition to the process according to the first embodiment, a process of proposing a travel plan with the location identified by the user as the destination can also be performed. For example, the travel suggestion device 100A according to the second embodiment can suggest a travel plan to an image viewer who selects a fictitious landscape image posted on an SNS, with the destination being a location where a landscape similar to the image can be seen.

[0055] (Overall composition) 8 is a diagram showing an example of the overall configuration of a system according to embodiment 2. In the travel suggestion system 10 according to embodiment 2, a travel suggestion device 100A, which is the information processing device 100 according to embodiment 1 to which components such as an image accepting unit 126 and a travel plan presenting unit 128, which will be described later, are added, a place name search server 200, a viewer terminal 300-1 and a contributor terminal 300-2, which are examples of the user terminal 300 according to embodiment 1, and a travel agency server 400 are communicably connected via a network.

[0056] The travel suggestion device 100A is an information processing device in which components for realizing a function of proposing a trip to an image viewer, such as an image accepting unit 126 that accepts information about fictitious scenery images from an image viewer, and a travel plan presenting unit 128 that presents a travel plan with the identified places as destinations, are added to the server device that performs the process of identifying place names where scenery similar to a received image can be seen, as described in the first embodiment. Furthermore, the travel suggestion device 100A transmits and receives information to and receives information from 300-1, poster terminal 300-2, and travel agency server 400 via a dedicated application (hereinafter referred to as a travel app) that supports the processing of an image posting unit 125 and a reward granting unit 129, which will be described later.

[0057] Viewer terminal 300-1 is a terminal owned by a user to whom a travel plan is presented by travel suggestion device 100 A. For example, viewer terminal 300-1 displays images posted to an SNS, which is a communication tool used over the Internet, or a travel plan generated by a travel plan presentation step described later.

[0058] Contributor terminal 300-2 is a terminal owned by a user who submits fictional scenery images to be accepted by travel suggestion device 100 A. For example, contributor terminal 300-2 has a travel app installed, and posts fictional scenery images taken in a game using the application to an SNS, and receives rewards that are given when an image viewer completes a trip.

[0059] The travel agency server 400 is a server device that generates a travel plan with the identified location as the destination based on information on the place name of the location identified by the travel suggestion device 100A and information indicating the travel content preferences of the image viewer input into the viewer terminal 300-1, and is realized by a computer, a cloud system, etc.

[0060] For example, when the travel agency server 400 is notified of information about the place name of a location identified by the travel suggestion device 100A, it creates multiple travel plans with the identified location as the destination and displays the multiple generated travel plans on the viewer terminal 300-1.

[0061] (Outline of the process of the travel suggestion device 100A) Next, the processing contents of the travel suggestion device 100A according to the second embodiment will be explained. The travel suggestion device 100A receives information about a fictitious landscape image from the image viewer and identifies place names where a landscape similar to the landscape in the image can be seen. Then, the travel suggestion device 100A presents the image viewer with a travel itinerary created with the place indicated by the identified place name as a destination.

[0062] Here, an outline of the processing of the travel suggestion device 100A will be explained with a specific example using Fig. 9. Fig. 9 is a diagram illustrating an outline of the processing of the travel suggestion device according to embodiment 2. For example, the travel suggestion device 100A receives from the viewer terminal 300-1 a selection of a fictional landscape image in a game posted to the SNS site A accessed by the viewer terminal 300-1.

[0063] Next, the travel suggestion device 100A performs the series of processes according to the first embodiment described above on the received fictional scenery image, and identifies the place name of a location where scenery similar to the received fictional scenery image can be seen. Then, the travel suggestion device 100A displays information on the place name indicated by the identified location and a scenery photo of the identified location on the viewer terminal 300-1.

[0064] Next, when the travel suggestion device 100A receives a request from an image viewer to search for a travel plan, it displays on the viewer terminal 300-1 a plurality of travel plans with the viewer's previously set place of residence as the departure point and the identified location as the destination.

[0065] As a result, the travel suggestion device 100A can present a travel plan to an image viewer who has selected an imaginary landscape image posted on an SNS, with the travel plan having a destination of a place where a landscape similar to the selected landscape can be seen, and can easily suggest a travel plan to a place where a landscape similar to the landscape in the image can be seen, from an image depicting an imaginary landscape.

[0066] (Configuration of travel suggestion device 100A) An example of the functional configuration of the above-described travel suggestion device 100A will be described below. Fig. 10 is a block diagram showing an example of the configuration of a travel suggestion device according to embodiment 2. The travel suggestion device 100A has an image posting unit 125, an image accepting unit 126, a travel plan presenting unit 128, and a reward granting unit 129 in addition to the place name identifying unit 127 having the estimation unit 121, adjustment unit 122, generation unit 123, and identification unit 124 according to embodiment 1.

[0067] The image posting unit 125 posts the image depicting the fictional scenery received from the image poster to a communication tool via the Internet. For example, the image posting unit 125 posts the fictional scenery image taken by the image poster in the game to a predetermined SNS via a travel app that the image poster has installed in advance on the poster terminal 300-2.

[0068] Here, the communication tool via the Internet to which the image is posted is not particularly limited as long as it is an SNS that allows a user to post any image selected by the user via poster terminal 300-2. Note that image posting unit 125 can display the above-mentioned information when an image is selected by, for example, adding a link to a URL (Uniform Resource Locator) of a site that identifies a location where a landscape similar to the fictional landscape image can be seen or a proposed travel plan to the image posted to the SNS.

[0069] Furthermore, the image posting unit 125 posts a set of an image taken by the image viewer at a destination in the travel plan and an image depicting a fictitious landscape to a communication tool via the Internet. For example, the image posting unit 125 first accepts landscape photos taken at a destination visited by the image viewer using a travel plan presented by the travel plan presenting unit 128 (described later) via a travel app previously installed on the viewer terminal 300-1. Then, the image posting unit 125 composes a set of the photo taken at the destination and the fictitious landscape image posted by the image poster, and posts the set to an SNS in response to a posting instruction input to the viewer terminal 300-1.

[0070] Here, the processing content of the image posting unit 125 will be described with reference to Fig. 11. Fig. 11 is a diagram showing an example of processing by the image posting unit according to embodiment 2. Fig. 11 shows a case where an image poster posts a fictitious landscape image, and a case where an image viewer who has traveled posts a set of a photo taken at a destination and a fictitious landscape image selected by the image viewer.

[0071] 11 is, for example, a fictional landscape image initially posted by an image poster, and is an image accepted as a processing target by the information processing device 100. In contrast, the control image is a landscape photograph taken with a similar composition to the cited image by an image viewer who traveled to the destination suggested by the cited image, and is a photograph of a landscape similar to the fictional landscape of the cited image.

[0072] First, the image posting unit 125 accepts a fictional landscape image "a" taken by an image poster in a game via a travel app "B" installed on the poster terminal 300-2, and posts it as a quoted image to the SNS site "A" (Figure 11(1)).

[0073] Next, the image viewer selects the fictional landscape image "a" taken in the game that was posted as a quoted image on the SNS site "A," adopts the presented travel plan, and travels to the specified location. The image viewer then visits a location where a landscape similar to the fictional landscape image "a" taken in the game can be seen, and takes a photo "b" of the real landscape with a similar composition.

[0074] Thereafter, the image posting unit 125 accepts, via the travel app "B" installed on the viewer terminal 300-1, a photograph "b" of a real landscape taken by the image viewer at the destination as a comparison image, and also accepts, as a reference image, a fictional landscape image "a" taken in a game selected by the image viewer, and posts both images as a set to the SNS site "A" (Figure 11(2)).

[0075] As a result, the image posting unit 125 can attract the interest of users of the SNS by posting a contrast image, which is a photograph of a real landscape, in contrast to the cited image, which is an image of a fictitious landscape.

[0076] The image accepting unit 126 accepts information related to a fictional landscape image from an image viewer. For example, when a quoted image posted on an SNS site accessed by the viewer terminal 300-1 is tapped by the image viewer, the image accepting unit 126 accepts the tapped image as an image to be subjected to a series of processes of the information processing device according to the first embodiment described above. Note that the image accepting unit 126 can accept image data from an image poster by presenting a link to image data indicating the fictional landscape image or by accepting the image data of the fictional landscape image itself.

[0077] Here, the image receiving unit 126 can also receive the selection of a comparison image. In this case, the travel suggestion device 100A can propose to the image viewer a travel plan with the shooting location of the comparison image as the destination, without performing the series of processes according to the first embodiment described above, for example.

[0078] The place name identification unit 127 identifies the place name of a place where a landscape similar to the landscape of the fictional landscape image accepted by the image accepting unit 126 can be seen. For example, the place name identification unit 127 sequentially performs the processes of the estimation unit 121, adjustment unit 122, generation unit 123, and identification unit 124 according to the first embodiment on the image accepted by the image accepting unit 126, thereby identifying the place name of a place where a landscape similar to the fictional landscape image accepted from the viewer terminal 300-1 can be seen. Then, the travel suggestion device 100A displays, for example, the identified place name and a landscape photo of the place searched for during identification on the viewer terminal 300-1.

[0079] The processing by the place name identification unit 127 described above can be applied to fictional scenery images before they are posted to the SNS. For example, the travel suggestion device 100A can accept a fictional scenery image taken by an image poster in a game, perform a series of processes related to identifying a location for the accepted image, and grasp the result of identifying a location for the fictional scenery image before it is posted. This allows the image poster to select an image that is highly similar to the photo of the identified location from among multiple fictional scenery images taken by the image poster before posting, and post the image to the SNS.

[0080] The travel plan presentation unit 128 presents the image viewer with the travel plan created with the identified location as the destination. For example, when the image viewer inputs a request to search for a travel plan to the location identified by the series of processes described above, the travel plan presentation unit 128 notifies the travel agency server 400 of information on the identified location and causes the viewer terminal 300-1 to display multiple travel plans created by the travel agency server 400.

[0081] Here, the travel agency server 400 is notified in advance via the travel app of, for example, the image viewer's desired travel content information, information on the departure point, etc., and can generate multiple travel plans based on the notified information. Specifically, the travel agency server 400 obtains information required to create a travel plan from the image viewer, such as the name of the departure point, the number of people wishing to travel, the means of transportation, and the number of travel days, and generates multiple travel plans based on the obtained information.

[0082] The reward granting unit 129 grants a predetermined reward to the image poster when the image viewer travels according to the travel plan presented by the travel plan presenting unit 128. For example, when it is confirmed that the image viewer has traveled according to the proposed travel plan, the reward granting unit 129 transmits, via a travel app, data of points that constitute the reward to the poster terminal 300-2 of the image poster who posted the image selected by the image viewer to the SNS.

[0083] Here, the reward posted to poster terminal 300-2 is not particularly limited, and may be points, electronic coupons, or products or cash when a certain amount of reward is accumulated. In addition, the reward is given to the image poster, for example, by travel agency server 400 transmitting the reward to information processing device 100, and information processing device 100 receiving the reward transmits the reward to poster terminal 300-2 via a travel app.

[0084] (Example) Next, the overall processing flow of the travel suggestion device 100A according to the second embodiment will be described with reference to Fig. 12 to Fig. 14. Fig. 12 is a diagram showing a specific example of processing by an image posting unit and an image receiving unit according to the second embodiment. Fig. 13 is a diagram showing a specific example of processing by a place name identification unit and a travel plan presenting unit according to the second embodiment. Fig. 14 is a diagram showing a specific example of processing by an image posting unit and a reward granting unit according to the second embodiment.

[0085] As shown in Fig. 12, the travel suggestion device 100A first accepts a fictional landscape image "a" taken in a game by an image poster via a travel app "B" installed on the poster terminal 300-2, and posts the accepted image to the SNS site "A" (Fig. 12(1)). Next, when an image viewer accesses the SNS site "A" using the viewer terminal 300-1 and taps on the posted landscape image "a," the travel suggestion device 100A accepts the tapped image as the image selected by the image viewer (Fig. 12(2)).

[0086] Next, the travel suggestion device 100A adjusts the prompt sentence estimated by the reverse prompt for the received scenic image "a", and then searches the place name search server 200 for an image generated by the adjusted prompt sentence, and identifies the actual scenic photo "b", the shooting location "XX", and the location information "···", and displays them on the viewer terminal 300-1. Next, the travel suggestion device 100A accepts input to an input button indicating that a travel plan to the identified location is to be searched for (FIG. 13(3)).

[0087] Then, the travel suggestion device 100A notifies the travel agency server 400 of the location information "···" of the identified location "○○", the image viewer's desired travel content information, information on the departure point, etc., and displays the contents of the travel plans "Plan 1" and "Plan 2" created based on the notified information on the viewer terminal 300-1 (Figure 13(4)).

[0088] Next, while the image viewer is traveling according to the proposed itinerary, the travel suggestion device 100A accepts a landscape photo "c" taken with a similar composition to the landscape image "a" via the travel app "B" installed on the viewer terminal 300-1, and posts the accepted landscape image "c" and the landscape image "a" as a set to the SNS site "A" (Figure 14(5)).

[0089] Thereafter, if the travel suggestion device 100A determines that the image viewer has undertaken a trip based on a travel suggestion based on the scenic image "a," it will award a reward of "1000 points" to the travel app "B" installed on the poster terminal 300-2 (Figure 14(6)).

[0090] Through the series of processes described above, the travel suggestion device 100A identifies real locations where scenery similar to that of a fictional landscape image posted on an SNS can be seen, and proposes a travel plan with the identified location as the destination, thereby being able to propose a travel plan to a location where scenery similar to the fictional landscape image can be seen.

[0091] (flowchart) Next, an example of processing by the travel suggestion device 100A according to the second embodiment will be described with reference to Fig. 15. Fig. 15 is a flowchart showing an example of the flow of processing by the travel suggestion device according to the second embodiment. Note that the steps in the flowchart shown in Fig. 15 may be executed in a different order, and some processing may be added or omitted.

[0092] First, the travel suggestion device 100A accepts fictional scenery images taken by an image poster in the game and posts the images to an SNS (S201). Next, the travel suggestion device 100A accepts fictional scenery images posted to an SNS selected by an image viewer (S202).

[0093] Next, the travel suggestion device 100A identifies locations where scenery similar to the received fictitious scenery image can be seen (S203).The travel suggestion device 100A then generates an itinerary with the identified locations as destinations and presents the itinerary to the image viewer (S204).The image viewer then purchases the generated itinerary (S205) and travels (S206).

[0094] Then, the travel suggestion device 100A accepts photos taken by the image viewer at the destination, and posts the accepted photos and the fictional landscape image posted by the image poster as a set to the SNS (S207). After that, the travel suggestion device 100A gives a reward to the image poster based on the image viewer's completion of the trip (S208), and ends the process.

[0095] (effect) The travel suggestion device 100A according to the second embodiment includes an image receiving unit 126, a place name identification unit 127, and an itinerary presentation unit 128. The image receiving unit 126 receives information about a predetermined landscape image from the image viewer. The place name identification unit 127 identifies a location where a landscape similar to the landscape of the image received by the image receiving unit 126 can be seen. The itinerary presentation unit 128 presents to the image viewer a travel itinerary that has been generated with the location identified by the place name identification unit 127 as the destination.

[0096] As a result, the travel suggestion device 100A can present an image viewer who has selected a fictitious landscape image with a travel plan that sets a destination at a place where a landscape similar to the selected image can be seen, and therefore can easily suggest a travel plan to a place where a landscape similar to the landscape in the image can be seen from an image depicting a fictitious landscape.

[0097] The travel suggestion device 100A further includes an image posting unit 125. The image posting unit 125 posts images depicting fictional scenery received from image posters to a communication tool via the Internet. In this case, the image accepting unit 126 accepts a selection of images depicting fictional scenery posted to the communication tool via the Internet.

[0098] As a result, when an image viewer using an SNS selects a posted fictitious landscape image, the travel suggestion device 100A can present a travel plan with destinations at places where landscapes similar to the selected landscape can be seen.

[0099] The image posting unit 125 posts a set of an image taken by the image viewer at the destination and an image depicting the fictional scenery to a communication tool via the Internet. This allows the travel suggestion system 10 to post real scenery photos in contrast to the fictional scenery images, thereby attracting the interest of users of the SNS.

[0100] The travel suggestion device 100A further includes a reward granting unit 129. When the image viewer travels according to the travel plan presented by the travel plan presenting unit 128, the reward granting unit 129 grants a predetermined reward to the image contributor who posted the scenic image selected by the image viewer to a communication tool via the Internet.

[0101] This allows the travel suggestion device 100A to reward the image poster depending on the image viewer's record of going on a trip, thereby encouraging the image poster to post fictional scenery images to the SNS.

[0102] 〔others〕 Furthermore, among the processes described in the above embodiments, all or part of the processes described as being performed automatically may be performed manually. Conversely, all or part of the processes described as being performed manually may be performed automatically using a known method. In addition, the information including the processing procedures, specific names, various data, and parameters shown in the above documents and drawings may be changed as desired unless otherwise specified.

[0103] Furthermore, the components of each device shown in the figures are conceptual functional units and do not necessarily have to be physically or functionally configured as shown. In other words, the specific form of distribution and integration of each device and functional configuration is not limited to that shown in the figures, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc. Furthermore, the above-mentioned embodiments and the processes performed in each embodiment may be combined as appropriate within the scope of not causing any contradiction in the processing content.

[0104] 〔program〕 The information processing device 100 according to the first embodiment or the travel suggestion device 100A according to the second embodiment described above is realized by, for example, a computer 1000 configured as shown in Fig. 16. Fig. 16 is a diagram showing an example of a computer that executes a program according to the first embodiment or the second embodiment. The computer 1000 is connected to an output device 1010 and an input device 1020, and has a configuration in which a central processing unit 1030, a memory 1040, a storage 1050, an output IF (Interface) 1060, an input IF 1070, and a communication interface 1080 are connected via a bus 1090.

[0105] The central processing unit 1030 operates based on programs stored in the memory 1040 or storage 1050, programs read from the input device 1020, and the like, and executes various processes. The memory 1040 is a memory device, such as a RAM, that temporarily stores data used by the central processing unit 1030 for various calculations. The storage 1050 is a storage device in which data used by the central processing unit 1030 for various calculations and various databases are registered, and is realized by a ROM (Read Only Memory), an HDD (Hard Disk Drive), a flash memory, or the like.

[0106] The output IF 1060 is an interface for transmitting information to be output to an output device 1010 that outputs various types of information, such as a monitor or a printer, and is realized by a connector conforming to a standard such as USB (Universal Serial Bus), DVI (Digital Visual Interface), or HDMI (registered trademark) (High Definition Multimedia Interface). The input IF 1070 is an interface for receiving information from various input devices 1020, such as a mouse, keyboard, camera, etc., and is realized by a USB, etc. The input device 1020 may be an external storage medium such as a device that reads information from an optical recording medium, a magneto-optical recording medium, a tape medium, a magnetic recording medium, or a semiconductor memory, or a USB memory.

[0107] The communication interface 1080 receives data from other devices via the network N and sends it to the central processing unit 1030, and also transmits data generated by the central processing unit 1030 to other devices via the network N. The central processing unit 1030 controls the output device 1010 and the input device 1020 via the output IF 1060 and the input IF 1070. For example, the central processing unit 1030 loads a program from the input device 1020 or the storage 1050 onto the memory 1040 and executes the loaded program.

[0108] For example, when the computer 1000 functions as the information processing device 100 , the central processing unit 1030 of the computer 1000 executes a program loaded onto the memory 1040 to realize the functions of the control unit 120 .

[0109] Although some embodiments of the present application have been described above with reference to the drawings, these are merely examples, and the present invention may be embodied in other forms that incorporate various modifications and improvements based on the knowledge of those skilled in the art, including the aspects described in the Disclosure of the Invention section. Furthermore, the terms "section, module, unit" mentioned above can be read as "means" or "circuit," etc. For example, the term "applying section" can be read as "applying means" or "applying circuit." [Explanation of symbols]

[0110] 10 Travel recommendation system 100 Information processing device 100A Travel suggestion device 110 Communications Department 120 control section 121 Estimation Department 122 Adjustment section 123 Generation part 124 Specific part 125 Image Submission Section 126 Image Reception Department 127 Place name identification section 128 Travel Plan Presentation Section 129 Reward Division 130 Storage section 200 Place Name Search Server 300 User Terminals 300-1 Viewer terminal 300-2 Poster's terminal 400 Travel agency server

Claims

1. an estimation unit that estimates a prompt sentence for generating a fictional landscape image using a generative model; an adjustment unit that adjusts the prompt sentence estimated by the estimation unit so that an image generated by the prompt sentence becomes a realistic image; a generation unit that generates an image using a generative model by using the prompt sentence adjusted by the adjustment unit; An information processing device comprising:

2. 2. The information processing apparatus according to claim 1, further comprising an identifying unit that identifies a location where an image of a real scene similar to the image generated by the generating unit was taken.

3. The information processing apparatus according to claim 1 , wherein the adjustment unit deletes, from the estimated prompt sentence, a phrase that specifies a property of an image to be generated from the prompt sentence.

4. The information processing device according to claim 2, characterized in that the identification unit searches for the image generated by the generation unit on a predetermined terminal that stores a set of an image of the real scenery and the location where the image of the real scenery was taken.

5. The information processing device according to claim 2 , wherein the identification unit identifies the shooting location from the prompt sentence estimated for an image of a real landscape similar to the image generated by the generation unit.

6. An information processing method executed by an information processing device, an estimation step of estimating, for a given image, a prompt sentence for generating said image using a generative model; an adjustment step of adjusting the prompt sentence estimated by the estimation step so that an image generated by the prompt sentence becomes a realistic image; a generation step of generating an image using a generative model using the prompt sentence adjusted by the adjustment step; An information processing method comprising:

7. 7. The information processing method according to claim 6, further comprising a step of identifying a location where an image of a real scene similar to the image generated by said generating step was taken.

8. an estimation step of estimating, for a given image, a prompt sentence for generating said image using a generative model; an adjustment step of adjusting the prompt sentence estimated by the estimation step so that an image generated by the prompt sentence becomes a realistic image; a generation step of generating an image using a generative model by using the prompt sentence adjusted by the adjustment step; An information processing program that causes a computer to execute the above.

9. 9. The information processing program according to claim 8, further comprising a step of causing a computer to execute a step of identifying a location where an image of a real landscape similar to the image generated by the step of generating the image is taken.

Images