Information processing device, information processing method, program, and recording medium

Abstract

Provided are an information processing device, an information processing method, a program, and a recording medium capable of supporting capture and generation of images conforming to a user's desired concept. An information processing device 100 according to one embodiment of the present invention comprises a processor 11 which executes: processing for accepting input of an image Pf; processing for analyzing the image Pf and generating information in natural language as information relating to modification of the image Pf; and processing for outputting the information in natural language.

Description

Information Processing Apparatus, Information Processing Method, Program, and Recording Medium

[0001] One embodiment of the present invention relates to an information processing apparatus, an information processing method, a program, and a recording medium that enable photographing and generating an image along a user-desired image.

[0002] With the evolution and spread of digital cameras, the opportunities and needs for photography have increased not only for professionals but also for ordinary individuals. From the perspective of image acquisition, it has become possible to easily generate an image by inputting a prompt according to the image of a desired image into so-called generative AI (Artificial Intelligence). On the other hand, although it has become possible to easily acquire an image in this way, there are cases where the image does not conform to the desired image. Therefore, techniques for processing and editing an image according to the user's purpose and the like are already known.

[0003] As an example of such a technique, the technique described in Patent Document 1 can be cited. In Patent Document 1, in an information processing apparatus in which a user applies a predetermined process to an image to process the image, a technique for intuitively approaching the intended image quality without the user needing specialized knowledge of image processing is disclosed.

[0004] This technique includes an image feature extraction means for analyzing an input image and extracting a feature amount, a reference image storage means for storing in association a reference image prepared in advance, a feature amount extracted from the reference image using the image feature extraction means, a result image that is a result of processing the reference image, the image processing used for the processing, and the parameters of the image processing used for the processing, a reference image search means for searching the reference image storage means using the feature amount extracted from the input image and acquiring a reference image similar to the input image, an image display means for displaying a set of the result image corresponding to the reference image acquired by the reference image search means, a reference image selection means for selecting an arbitrary set from the set displayed by the image display means, and an image processing means for processing the input image using the parameters of the image processing corresponding to the set selected by the reference image selection means. It relates to an information processing apparatus characterized by comprising these.

[0005] Furthermore, Patent Document 2 discloses an information processing method and apparatus that can change the color temperature of an image to achieve a desired impression, as well as a computer-readable recording medium that stores a program for causing a computer to execute the information processing method.

[0006] This technology relates to an information processing method for performing color conversion on an image having a predetermined color temperature for output to an output medium, characterized in that the color temperature of the image is changed based on specified information for changing the predetermined color temperature, and the color conversion is performed on the image whose color temperature has been changed.

[0007] Japanese Patent Publication No. 2012-146071 Japanese Patent Publication No. 2000-261825

[0008] It is certainly possible to edit images obtained through photography or AI generation using image editing applications, and to change various parameters such as color temperature and white balance. On the other hand, in order to efficiently obtain the desired editing results by changing such parameters, knowledge and experience regarding the correspondence between those parameters and the image after editing are necessary. In particular, when manually setting the above parameters on a digital camera during shooting, detailed knowledge and proficiency in the camera's specifications and operation methods are also required.

[0009] The problems described above also apply to image generation using AI. If the image generated by the AI ​​differs from the image desired by the user, it becomes necessary to change or add prompts. However, accurately predicting the impact of such prompt changes on the generated image requires specialized knowledge and experience in AI generation. In other words, for those lacking the appropriate knowledge and experience, efficiently capturing and generating images that match the desired image is currently difficult.

[0010] One embodiment of the present invention has been made in view of the above circumstances and aims to provide a technology that can assist in capturing and generating images that conform to the image desired by the user.

[0011] The above objective is achieved by an information processing device described in any of the following [1] to

[18] . [1] An information processing device comprising a processor that makes suggestions regarding changes to an image, wherein the processor performs the following processes: receiving an image input; analyzing the image and generating information in natural language as information regarding changes to the image; and outputting the information in natural language.

[0012] [2] The information processing device described in [1], wherein the processor generates information corresponding to the modified image when a change is made to the image, as information in natural language.

[0013] [3] The information processing device described in [1], wherein the information in natural language is a natural language expression that includes the intention to change the image.

[0014] [4] The information processing device described in [1], wherein the information in natural language includes the intention to change the image and is expressed in natural language that is understandable to the user.

[0015] [5] The information processing apparatus according to [2], wherein the processor generates one or more groups containing two or more words as candidates for the modified image as information about the modification.

[0016] [6] The information processing device described in [1] outputs information regarding settings for acquiring a modified image when a change is made to an image, based on information in natural language.

[0017] [7] The information processing device described in [6], which outputs information of multiple groups of candidate images for the modified image, accepts the user's selection of the image to be modified from the multiple groups, and configures the image capture device based on the words of the group.

[0018] [8] The information processing apparatus described in [1] outputs information regarding settings for making changes to an image and generating a modified image based on information in natural language.

[0019] [9] The information processing apparatus according to [5], which outputs information of multiple groups of candidate images for the modified image, accepts the user's selection of the image to be modified from the multiple groups, and generates a modified image in the generation AI based on the words of the group.

[0020]

[10] The information processing apparatus according to [1], wherein the processor analyzes an image, extracts words that represent the image, and generates information about the changes based on those words.

[0021]

[11] The information processing device described in [1], wherein the processor accepts user input on the output information in natural language.

[0022]

[12] The information processing apparatus according to [5], wherein the processor accepts a modification operation for at least one of two or more words included in the group.

[0023]

[13] The information processing device described in

[12] , wherein the modification operation is an operation to input information regarding the priority of two or more words included in a group.

[0024]

[14] The information processing device described in

[12] , wherein the modification operation is an operation to input information to modify two or more words included in a group.

[0025]

[15] The information processing device according to

[11] , wherein the processor accepts information for two or more words that belong to two or more groups from the output natural language information as a user modification operation.

[0026]

[16] The information processing device described in [1], wherein the processor stores the generated natural language information in association with an image.

[0027]

[17] The information processing apparatus according to

[16] , wherein the processor regenerates the modified image when a change is made to the image, based on any of the natural language information stored regarding the image.

[0028]

[18] The information processing apparatus according to [1], which determines, based on information in natural language, to change or add at least one of the following as changes to an image: color, brightness, and subject matter.

[0029] Furthermore, the above objective can also be achieved by the information processing method described in

[19] below.

[19] An information processing method comprising: a processor that accepts an image as input; a processor that analyzes the image and generates information in natural language as information relating to changes in the image; and a processor that outputs the information in natural language.

[0030] Furthermore, the above objectives can also be achieved by the program described in

[20] below:

[20] A program for causing a computer to execute each step included in the information processing method described in

[19] .

[0031] Furthermore, the above objectives can also be achieved by the recording medium described in

[21] below.

[21] A computer-readable recording medium on which a program is recorded causing a computer to perform each step of the information processing method described in

[19] .

[0032] According to one embodiment of the present invention, an information processing device, an information processing method, a program, and a recording medium are provided that can assist in capturing and generating images in accordance with an image desired by the user.

[0033] This figure shows an example configuration of the information processing system according to this embodiment. This figure shows an example hardware configuration of the information processing device according to this embodiment. This figure shows the functional part of the information processing device according to this embodiment. This figure shows an example input image table according to this embodiment. This figure shows an example text information table according to this embodiment. This figure shows an example image definition table according to this embodiment. This figure shows an example setting item table according to this embodiment. This figure shows an example image table according to this embodiment. This figure shows an example flow of the information processing method according to this embodiment. This figure shows an example screen according to this embodiment. This figure shows an example screen according to this embodiment. This figure shows an example screen according to this embodiment. This figure shows an example screen according to this embodiment. This figure shows an example screen according to this embodiment. This figure shows an example screen according to this embodiment. This figure shows an example screen according to this embodiment. This figure shows an example screen according to this embodiment.

[0034] The following describes specific embodiments of the present invention. For convenience of explanation, the following descriptions may sometimes be based on the perspective of a GUI (Graphical User Interface). Furthermore, since the fundamental data processing technologies for realizing the present invention (communication / transmission technologies, data acquisition technologies, data recording technologies, data processing / analysis technologies, machine learning technologies, image processing technologies, and visualization technologies, etc.) are known technologies, their descriptions will be omitted.

[0035] Furthermore, in this specification, the concept of "device" includes not only a single device that performs a specific function, but also a combination of multiple devices that exist independently and in a distributed manner while cooperating (linking) to perform a specific function.

[0036] Furthermore, in this invention, "user" refers to a user of the information processing device of the present invention, and specifically, for example, a person who performs appropriate parameter settings for a digital camera or appropriate prompt input in an image generation AI using the functions of the information processing device of the present invention.

[0037] Furthermore, in this specification, "person" means an entity that performs a specific action, and includes individuals, groups, corporations and other legal entities, and organizations, and may also include computers and devices that constitute artificial intelligence (AI). Artificial intelligence (AI) realizes intelligent functions such as reasoning, prediction, and judgment using hardware and software resources. The algorithm of artificial intelligence is arbitrary and includes, for example, expert systems, case-based reasoning (CBR), convolutional neural networks (CNN), deep neural networks (DNN), Bayesian networks, or inclusion architectures.

[0038] <<About one embodiment of the present invention>> [Configuration of the information processing system] In one embodiment of the present invention (hereinafter, this embodiment), the information processing system 5 is configured by an information processing device 100 and a generation AI server 400 connected to the network N shown in Figure 1. The information processing device 100 is composed of at least one of a shooting device 200 and a user terminal 300.

[0039] Of these, the shooting device 200 is a device for the user to capture an image (hereinafter referred to as the initial image Pf) as needed. The initial image Pf is an image that can be modified in terms of its image and shooting parameters, etc., according to the user's preferences and circumstances. In this embodiment, the initial image Pf captured by the shooting device 200 will be given as an example of a live view image from a digital camera (of course, it is not limited to this form).

[0040] Furthermore, the imaging device 200 executes the information processing method of this embodiment and presents to the user, in natural language, a proposed policy for modification processing (image modification) of the initial image Pf, and obtains the selection result, etc. Depending on this selection result, the imaging device 200 performs the corresponding modification processing either by itself or in appropriate cooperation with the generation AI server 400 or the user terminal 300, and obtains the modified image Ps. Specifically, such an imaging device 200 is a digital camera. The modification processing can be either a change in the live view image due to a change in the imaging parameters of the imaging device 200, or an image regeneration due to a change in the prompt Tp in the image generation AI 211.

[0041] On the other hand, the user terminal 300 may have the same configuration and functions as the imaging device 200, but its minimum configuration would be to obtain a prompt Tp from the user and input it into its own image generation AI 211 (described later in Figure 3) or send it to the generation AI server 400 to generate the initial image Pf and the modified image Ps. Of course, the user terminal 300 may also be equipped with an imaging unit similar to the imaging device 200 and capture the initial image Pf as needed.

[0042] Furthermore, the user may, for example, scan or read data such as a printed photograph or the screen of their user terminal 300 (showing a photograph) as the initial image Pf into the UI (User Interface) of the information processing device 100, such as the camera 200 or user terminal 300, or they may directly input the prompt Tp using a keyboard, mouse, touch panel, etc., which are part of the UI. If such a configuration and operation is adopted, the information processing system 5 can consist only of the information processing device 100.

[0043] The above-mentioned shooting device 200 and user terminal 300 store the initial image Pf and modified image Ps obtained from shooting and image generation in the input image table 201 (described later in Figures 3 and 4), and the prompt Tp in the text information table 202 (described later in Figures 3 and 5), respectively, in preparation for changing the settings of shooting parameters, providing information for image regeneration by the image generation AI 211, and various other processes.

[0044] The information processing device 100 is an information processing device that executes processes such as generating or modifying the initial image Pf, or generating (regenerating) the modified image Ps, or changing the shooting parameters in the imaging device 200, using the image generation AI and the caption generation AI that it possesses or that the generation AI server 400 provides as a function.

[0045] Also, as described above, the generation AI server 400 is a server device that provides the functions of the image generation AI and the caption generation AI to the information processing device 100 via the network N. Of course, if the information processing device 100 is configured not to require the provision of the functions of the image generation AI and the caption generation AI, this generation AI server 400 may not be included in the information processing system 5.

[0046] In addition to the form in which the information processing device 100 and the generation AI server 400 are connected by the network N, a form in which the internal bus wiring of the information processing device 100 and the interface of the generation AI server 400 are directly connected may also be used.

[0047] Also, various situations can be assumed, such as cases where the photographer who performs various operations related to shooting and outputting photo prints by the imaging device 200, the registrant who inputs the image (initial image Pf) obtained by the shooting and its photo print to the user terminal 300 (including a scanning operation or an upload from the imaging device 200), and the viewer who views the initial image Pf, the modified image (correction image candidate group), the modified image Ps, etc., which are the processing results of the information processing device 100, on the imaging device 200 or the user terminal 300, are all different people or all the same person.

[0048] In addition, the "image" in the present invention is composed of a plurality of pixels, expressed by the gradation values of each of the plurality of pixels, and includes an image of at least one subject. Further, digital image data (hereinafter referred to as image data) that defines an image at a set resolution is generated by compressing data in which gradation values for each pixel are recorded using a predetermined compression method. Examples of the types of image data include non-reversible compression image data such as JPEG (Joint Photographic Experts Group) format, and reversible compression image data such as GIF (Graphics Interchange Format) or PNG (Portable Network Graphics) format, etc.

[0049] [Configuration Example of Information Processing Apparatus] Next, a configuration example of the information processing apparatus 100 according to the present embodiment will be described while referring to FIGS. 2 and 3. The information processing apparatus 100 consists of a computer used by a user (including the concept of the computer chip of the imaging apparatus 200), and specifically, it is composed of a digital camera, a PC (Personal Computer), a smartphone, a tablet terminal, a notebook PC, or the like. Note that the information processing apparatus 100 is not limited to a computer owned by the user, and may be composed of a terminal that can be used when visiting a store or facility that the user does not own, such as a store-installed terminal. Hereinafter, a case where the information processing apparatus 100 is composed of the user-owned imaging apparatus 200 or a PC will be described as an example.

[0050] As shown in FIG. 2, the computer constituting the information processing apparatus 100 includes a processor 11, an auxiliary storage device 12, a main storage device 13, an input device 14, an output device 15, and a communication device 16.

[0051] The processor 11 is composed of, for example, a CPU (Central Processing Unit), an MPU (Micro-Processing Unit), an MCU (Micro Controller Unit), a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), a TPU (Tensor Processing Unit), or an ASIC (Application Specific Integrated Circuit).

[0052] The auxiliary storage device 12 is composed of, for example, flash memory, HDD (Hard Disk Drive), SSD (Solid State Drive), FD (Flexible Disk), MO disk (Magneto-Optical Disk), CD (Compact Disk), DVD (Digital Versatile Disk), SD card (Secure Digital card), or USB memory (Universal Serial Bus memory).

[0053] Such auxiliary storage devices 12 may be built into the computer main body that constitutes the information processing device 100, or they may be attached to the computer main body as an external device. Alternatively, the auxiliary storage device 12 may be configured as a NAS (Network Attached Storage) or the like. Furthermore, the auxiliary storage device 12 may be an external device that can communicate with one of the computers that constitutes the information processing device 100 via a communication network, such as an online storage or database server.

[0054] Furthermore, the auxiliary storage device 12 holds the operating system (OS) and programs 121 such as applications related to image generation and modification. When the programs 121 are read and executed by the processor 11, the computer constituting the information processing device 100 performs the functions of the reception unit 20, generation unit 21, determination unit 22, and output unit 23 shown in Figure 3. Specifically, it performs a series of processes such as generating an initial image Pf, outputting various information related to the modification of the initial image Pf, and generating a modified image Ps in accordance with the modification of the initial image Pf. In other words, the processor 11 of the information processing device 100 has the above-mentioned reception unit 20, generation unit 21, determination unit 22, and output unit 23.

[0055] The main memory 13 is composed of semiconductor memory such as ROM (Read Only Memory) and RAM (Random Access Memory). The processor 11 loads the program 121 onto the main memory 13 and executes it there.

[0056] The input device 14 is a device that accepts user input and consists of, for example, a keyboard, mouse, or touch panel. The input device 14 may also include a digital camera unit and a microphone for sound collection. The output device 15 consists of, for example, a display or speaker. The digital camera unit in the input device 14 may be used in the same way as the shooting device 200.

[0057] The communication device 16 may be configured as, for example, a network interface card or a communication interface board. The computer constituting the information processing device 100 can communicate with other devices connected to a network N, such as the Internet and mobile communication lines, via the communication device 16.

[0058] As shown in Figure 3, the information processing device 100 includes a reception unit 20, a generation unit 21, a determination unit 22, and an output unit 23. These functional units are realized when the processor 11 of the information processing device 100 executes the aforementioned program 121 and cooperates with other hardware devices of the information processing device 100. In addition, the image generation AI 211 and caption generation AI 212 in the generation unit 21 may be called and executed from the generation AI server 400.

[0059] (Reception Unit) The reception unit 20 acquires the initial image Pf and modified image Ps from the user terminal 300 via the camera 200, the camera unit of the input device 14, or the communication device 16, and stores them in, for example, the auxiliary storage device 12. The reception unit 20 stores the initial image Pf, etc., acquired as described above in the input image table 201. The reception unit 20 acquires a prompt Tp from the keyboard of the input device 14 or from the user terminal 300 via the communication device 16 for the purpose of generating the initial image Pf, etc., or for the purpose of regenerating the initial image Pf, etc., for modification, and stores this in the text information table 202.

[0060] Figure 4 shows an example of the configuration of the input image table 201 in this embodiment. The input image table 201 is a collection of records containing the values ​​of image ID, date and time, image data, caption, and shooting parameters. Of these, the image ID is identification information that uniquely identifies an image, such as the initial image Pf. The date and time is the date and time the image was taken or acquired. The image data is the data file of the image.

[0061] A caption is a keyword obtained by assigning the target image to the caption generation AI 212. Such captions are one or more words obtained by the reception unit 20 or the generation unit 21 assigning images obtained from the shooting device 200, the shooting unit of the input device 14, or the user terminal 300 to the caption generation AI 212. The caption generation AI 212 can be held by the reception unit 20 or made available for retrieval from the generation AI server 400 via the network N.

[0062] The caption generation AI 212 is an AI that includes a pre-built machine learning model for caption generation. This model, for example, identifies the features of an initial image Pf and generates a caption corresponding to those features. It is constructed by performing machine learning using previously acquired initial images Pf and their associated captions (for example, appropriate captions assigned by knowledgeable individuals) as training data.

[0063] The technology for obtaining captions from images is not limited to using the caption generation AI 212 as described above. For example, the correspondence between the image analysis results and captions may be stored as data, and the receiving unit 20 may select (generate) a caption by referring to that correspondence. The correspondence between the image analysis results and captions may be, for example, an association between image analysis results such as the color and pattern of the background area, the type, shape, size, and theme of the subject object, and captions such as "sunset," "sepia," "city," "countryside," "illumination," "wasteland," and "idol." In other words, the caption can be information that indicates the subject in the image or its image.

[0064] Furthermore, the shooting parameters are the values ​​of the parameters used when shooting with the shooting unit, etc., and include values ​​for parameters such as F-number, shutter speed, ISO sensitivity, color tone, and brightness. These shooting parameter values ​​are extracted from metadata associated with the target image or obtained from the shooting unit of the shooting device 200 or input device 14.

[0065] On the other hand, Figure 5 illustrates the text information table 202 in which the prompt Tp obtained by the reception unit 20 is stored. Figure 5 shows an example of the configuration of the text information table 202 in this embodiment. The text information table 202 is a collection of records containing an ID that uniquely identifies the prompt Tp, a date and time, and text information. The ID may be the identification information of the user who entered the prompt Tp, or the identification information of the user terminal 300. The date and time is the date and time when the prompt Tp was obtained.

[0066] The text information consists of user-inputted prompts Tp, stored, for example, by keyword. These prompts Tp are the text entered by the user via the UI when the user inputs using the input device 14 or user terminal 300, stored in phrases and words. In this storage process, the reception unit 20 may provide the morphological analysis engine with the input prompt Tp sentence to extract one or more words and store them in their respective fields. The morphological analysis engine may be held by the reception unit 20 or made available via the network N. The text information in the text information table 202, i.e., the words and phrases constituting the prompts Tp, may indicate the subject or image in the image generated by the prompt Tp.

[0067] The method for receiving the initial image Pf and prompt Tp in the reception unit 20 is not particularly limited, but includes the method of acquiring them by performing a camera unit read from a photographic print of an image taken by the user terminal 300 or the shooting device 200. In addition, the reception unit 20 may acquire images by downloading image data from an external device or a web server via the network N.

[0068] (Generation Unit) The generation unit 21 analyzes the initial image Pf (which conceptually may also include a modified image Ps based on the initial image Pf) obtained by the reception unit 20, and generates information in natural language as information regarding the modification of the initial image Pf. This information in natural language includes the impression after modification (a group of candidate corrected images) when the initial image Pf is modified, and information (natural language expression) that presents the intention of the modification. The initial image Pf may also include those generated by the generation unit 21 by assigning a prompt Tp to the image generation AI 211. Furthermore, such prompts Tp may also include those generated by the generation unit 21 by assigning the initial image Pf to the caption generation AI 212.

[0069] Furthermore, when the generation unit 21 generates information on the corrected image candidate group, i.e., the corrected image candidate group, from the natural language information mentioned above, it generates one or more groups containing two or more words as the corrected image candidate group. The generation of this corrected image candidate group is performed, for example, when the user half-presses the shooting button on the shooting device 200 to display a live view image of the desired subject.

[0070] In this case, the generation unit 21 inputs the live view image to the caption generation AI 212 and extracts a caption (for example, "1 girl", "background green woods", "pond"). In this case, the caption becomes a keyword indicating the subject in the live view image. The generation unit 21 stores this caption in the "Caption" column of the input image table 201, for example. Next, the generation unit 21 generates multiple sets of correction image candidates (sets of multiple words that suggest the image after processing the changes to the live view image), i.e., a group of correction image candidates, based on the caption (and the live view image itself).

[0071] Specific examples of the correction image candidates that make up the above correction image candidate group include, for example, text-based information that is easy for humans to visualize the change, such as "noise added," "nostalgic," "CG-like," "futuristic," "horror," and "dark." Therefore, a correction image candidate group (hereinafter referred to as "Gr" in the sense of a group of correction image candidates) that includes multiple such correction image candidates will have the following structure: Gr. A: noise added, nostalgic, ..., Gr. B: CG-like, futuristic, ..., Gr. C: horror, dark, ...

[0072] The generation unit 21 is equipped with an AI that determines a preferred group of correction image candidates for the initial image Pf by inputting at least one of the caption and the initial image Pf, or it can be called and used from an external system. This AI is, for example, an AI equipped with a model that has undergone deep learning using a set of training data consisting of images (e.g., live view images or temporary generated images) and their captions, and a set of preferred correction image candidates (a set of keywords suggesting them) defined for the image by an expert (e.g., an image editing specialist or a professional photographer). In other words, the generation unit 21 can generate a group of correction image candidates consisting of multiple keywords by inputting the caption and the initial image Pf to the AI.

[0073] In addition, the following can also be used as other learning methods for the AI ​​model described above. For example, a professional photographer or other knowledgeable person can create the best image (work) based on the training images (live view images or hypothetical generated images), and the difference between this image and the training images can be expressed using a set of keywords by the knowledgeable person or a designated third party. Then, the AI ​​is given this set of training images and the keyword set representing the difference as training data, and deep learning is performed. The generation unit 21 applies the initial image Pf, etc., to this AI, thereby efficiently generating the keyword set, i.e., the corrected image candidate set.

[0074] In addition to the AI-based approach described above, another approach can be adopted to identify a suitable corrected image candidate for the initial image Pf using the definition of the corrected image candidate (Figure 6: Image Definition Table 213). The Image Definition Table 213 is a table that defines a group of captions representing each corrected image candidate for each case. As shown in Figure 6, the Image Definition Table 213 is a collection of records that link the values ​​of the definition ID, image, and caption.

[0075] Of these, "image" is a keyword that represents a candidate for a corrected image. "Caption" is a set of keywords that represent the image in question, and is defined for each case that applies to the image. These sets of "image" and "caption" may be, for example, predefined by an expert, or generated by an AI that has undergone deep learning to understand the correspondence between the impression a person receives from an image (i.e., the image) and the caption obtained for that image.

[0076] When using such an image definition table 213, the generation unit 21 compares the caption obtained from the initial image Pf (held in the "Caption" column of the input image table 201) with the "Caption" column of each "Image" in the image definition table 213, and identifies "Cases" and "Images" associated with those cases in which a certain percentage or more of the keywords match (e.g., 60% or more of the number of keywords defined for each case). This identifies a suitable group of correction image candidates for the initial image Pf, for example, "Nostalgic" and "Noise Added".

[0077] The generation unit 21 stores information about the correction image candidate group generated as described above for each corresponding image. For this reason, the generation unit 21 stores information about the correction image candidate group in the "Candidate Group" column of the record in the image table 222, for example.

[0078] Furthermore, the generation unit 21 may, in addition to generating multiple "groups" of correction image candidates for the target initial image Pf and storing them in the "candidate group" column, use specific parameters as correction image candidates. Specifically, for example, as shown in the example of record "D05" in the setting item table 221 and record "P003" in the image table 222, use the "brightness" parameter as information for correction image candidates. In this case, the generation unit 21 is equipped with AI that determines a specific set of parameters as a preferred correction image candidate group for the initial image Pf by inputting at least one of the caption and the initial image Pf, or it is possible to call and use such AI from an external system.

[0079] The AI ​​described above is an AI equipped with a model that has undergone deep learning using a set of training data consisting of images (e.g., live view images or hypothetical generated images) and their captions, and preferred parameter candidates defined for the image by an expert (e.g., an image editing specialist or a professional photographer). In other words, the generation unit 21 can generate a group of corrected image candidates consisting of parameter candidates by inputting the caption and initial image Pf into the AI.

[0080] (Decision Unit) The decision unit 22 is a functional unit that presents information on, for example, multiple correction image candidate groups generated by the generation unit 21 with respect to the initial image Pf to the user via the output unit 23, and determines the one selected by the user from among the correction image candidate groups as the change. For this reason, the decision unit 22 displays, for example, the information on the correction image candidate groups, i.e., a list of groups consisting of keyword groups (as already mentioned, "Gr.") on the display of the shooting device 200 while the live view image is being displayed, and accepts group selection from the user who is the photographer. Alternatively, the user terminal 300 of the user who is generating the image displays the list of the above groups, and accepts group selection from the user in the same manner. The decision unit 22 stores the information on the group selected in this way, i.e., the correction image candidate group, in the "Selected Group" column of the image table 222. Note that the user may select multiple groups. Alternatively, the user may select a specific keyword (correction image candidate) from one group and a specific keyword (correction image candidate) from another group.

[0081] Furthermore, the determination unit 22 identifies information for making changes according to the correction image candidate selected by the user, that is, information for acquiring and generating the modified image Ps, from, for example, the setting item table 221 (see Figure 7). The setting item table 221 is a collection of records that link the definition ID, image, and setting value. Of these, "image" is a keyword that indicates the correction image candidate, as already mentioned. The "setting value" is the specific value of the camera parameters and prompt Tp when changing the initial image Pf based on the correction image candidate "image".

[0082] Among these, camera parameters may include parameters such as f-number, shutter speed, ISO sensitivity, color tone, and brightness. The prompt Tp is text information that is given to the image generation AI 211 to obtain the modified image Ps. In this case, the prompt Tp would be a predefined text, such as "Change the background to a sunset sky and change the overall tone of the subject image to sepia," which is suitable when changing the initial image Pf to "nostalgic."

[0083] The determination unit 22 sets the camera parameter value from the setting item table 221 to the control unit in the shooting device 200. As a result, when shooting operations are performed at subsequent timings, shooting will be performed with the said camera parameter, and the modified image Ps can be obtained. The shooting device 200 can also change the live view image to the modified image Ps. The determination unit 22 also inputs the prompt Tp from the setting item table 221 to the image generation AI 211. As a result, the image generation AI 211 generates the modified image Ps.

[0084] The determination unit 22 may also output the above-mentioned set values, such as camera parameters and prompt Tp, to the display of the shooting device 200 or user terminal 300 via the output unit 23. In this case, users familiar with camera operation and the use of the image generation AI 211 will be presented with information that they can directly understand regarding the change of the initial image Pf. Alternatively, for users unfamiliar with camera operation and the use of the image generation AI 211, this will help them understand and learn the specific parameters to be set in the shooting device 200 and the specific prompt Tp to be input to the image generation AI 211 when performing shooting or image generation according to the "image" in the future.

[0085] Furthermore, when the decision unit 22 presents a group of candidate corrected images to the user, it may accept a modification operation for at least one of the keywords constituting the group of candidate corrected images. This modification operation corresponds to an operation to input information regarding the priority of the target keyword in the group of candidate corrected images. Alternatively, the modification operation corresponds to an operation to input information to modify a user-specified keyword among the keywords constituting the group of candidate corrected images.

[0086] The decision unit 22 accepts these modification operations. For example, if the user selects "Gr. A" from the group of correction image candidates, and the element that the user prioritizes, that is, the element whose modification intensity they want to strengthen, is "noise addition" out of "nostalgic" and "noise addition," then the user inputs a directive via the input device 14 indicating that "noise addition" has a higher priority. The decision unit 22 accepts this priority directive and, for example, increases or decreases the degree of the values ​​that contribute to "noise addition" among the camera parameters and prompt Tp, compared to the default values ​​in the setting item table 221. This allows the modification tendency of the live view image and generated image to be more to the user's liking.

[0087] Furthermore, the decision unit 22 may, for example, assume that in "Gr. A," selected by the user from the group of corrected image candidates, the elements that the user wants to select, that is, the elements they want to adopt as changes, are "nostalgic" and "horror." In that case, the user inputs a correction instruction via the input device 14 to delete "noise addition" and add "horror." The decision unit 22 receives this deletion and addition instruction and, for example, deletes the value related to "noise addition" and adds the value related to "horror" from the camera parameters or prompt Tp. This allows the change tendency of the live view image and generated image to be modified to better suit the user's preferences.

[0088] Furthermore, the determination unit 22 re-executes the acquisition and generation of the modified image Ps, for example, when the initial image Pf has been modified, based on the corrected image candidate group that the user has re-selected from the multiple corrected image candidate groups or the corrected image candidate group that has been modified as described above. In addition, since the user re-selects from the multiple corrected image candidate groups, the determination unit 22 extracts the values ​​in the "Candidate Group" column of the image table 222 and displays the information of each group (Gr. A to) on the display so that it can be selected.

[0089] Furthermore, the decision unit 22 may determine the changes to the initial image Pf, etc., based on the user's selected group of corrected image candidates, including parameters such as F-number, shutter speed, ISO sensitivity, color tone, and brightness, as well as the change or addition of a subject. In this case, the decision unit 22 will display a UI on the output device 15 to receive user instructions regarding the change or addition of a subject, and will accept such instructions.

[0090] (Output Unit) The output unit 23 is a functional unit that displays the initial image Pf and modified image Ps obtained by the reception unit 20 or generation unit 21, as well as information on a group of correction image candidates which are candidates for changes to such images, on an output device 15 such as a display. For this purpose, the output unit 23 appropriately accesses the input image table 201, the text information table 202, and the image table 222, and appropriately retrieves and outputs information and data in response to user requests or the arrival of predetermined timings.

[0091] The output method in the output unit 23 is not particularly limited, but may include, for example, displaying various images and information on the output device 15, printing with a printer, transmitting to other users or systems, and providing as commercial products. The output format as commercial products may include media consisting of one or more pages or cards on which images are posted, such as albums, photobooks, postcards, message cards, electronic albums, and bromide prints.

[0092] [Example of Information Processing Method Flow] Next, as an example of the operation of the information processing device 100 in this embodiment, an information processing flow using the device will be described. The information processing method of the present invention is used in the flow described below. In other words, each step in the flow described below corresponds to a component of the information processing method of the present invention. Note that the following flow is merely an example, and some steps in the flow may be deleted, new steps added to the flow, or the execution order of two steps in the flow may be changed without departing from the spirit of this embodiment.

[0093] Each step in the image processing flow according to this embodiment is performed by the processor 11 of the information processing device 100 in the order shown in Figure 9. In other words, in each process in the flow, the processor 11 executes the data processing that corresponds to each step in Figure 9, from among the data processing defined in the application program for information processing.

[0094] To explain in more detail, in the flow according to this embodiment, first, the reception unit 20 acquires an initial image Pf from the user terminal 300 via the shooting unit of the shooting device 200 or input device 14, or via the communication device 16, and stores it in the input image table 201 (S1). An example of an initial image Pf obtained by such shooting is shown in screen G1 of Figure 10. Such an initial image Pf corresponds to the one displayed as a live view image on the shooting device 200, which is a digital camera.

[0095] In addition, the reception unit 20 may generate the initial image Pf using the image generation AI 211 instead of acquiring the initial image Pf by taking a photograph in S1 above. In that case, the reception unit 20 acquires the prompt Tp entered by the user who wishes to generate an image from the keyboard of the input device 14 or from the user terminal 300 via the communication device 16, and stores it in the "text information" column of the text information table 202 (Figure 5).

[0096] The text information stored in the text information table 202 consists of the prompts Tp entered by the user, stored, for example, in phrases or keywords. When storing this information, the reception unit 20 provides the sentence that constitutes the prompt Tp to the morphological analysis engine, which then extracts one or more phrases or words.

[0097] When the above prompt Tp is received, the reception unit 20 displays the screen G2 shown in Figure 11 on the output device 15 or the user terminal 300, and obtains the input value, prompt Tp, via the input field G21 of the screen G2. The reception unit 20 also receives a press of the generation button Bg on the screen G2 and assigns the prompt Tp to the image generation AI 211 to generate and obtain the initial image Pf. For example, the initial image Pf generated by assigning the prompt Tp "A tree-lined street in London with three-story brick buildings" as exemplified in Figure 11 to the image generation AI 211 could be the one shown in screen G1 of Figure 10.

[0098] Furthermore, the reception unit 20 obtains a caption by assigning the initial image Pf obtained in S1 to the caption generation AI 212 (S2), and stores it in the "Caption" column of the input image table 201 and the "Text Information" column of the text information table 202. However, the method of obtaining a caption from an image is not limited to using the caption generation AI 212 as described above. For example, the correspondence between the image analysis results and the caption may be stored as data, and the reception unit 20 may select (generate) a caption by referring to that correspondence. The correspondence between the image analysis results and the caption may be, for example, an association between image analysis results such as the color and pattern of the background area, the type, shape, size, and theme of the object being photographed, and captions such as "sunset," "sepia," "city," "countryside," "illumination," "wasteland," and "idol." In other words, the caption can be information that indicates the subject in the image or its image.

[0099] Furthermore, when acquiring the initial image Pf, the reception unit 20 acquires the values ​​of the shooting parameters used during shooting with the shooting unit, etc. (for example, F-number, shutter speed, ISO sensitivity, color tone, brightness, etc.) and stores these in the "Shooting Parameters" column of the input image table 201 (S3). These shooting parameter values ​​are extracted from metadata associated with the target image or obtained from the shooting unit of the shooting device 200 or input device 14.

[0100] The method for receiving the initial image Pf and prompt Tp in the reception unit 20 is not particularly limited, but includes the method of acquiring them by performing a camera unit read from a photographic print of an image taken by the user terminal 300 or the shooting device 200. In addition, the reception unit 20 may acquire images by downloading image data from an external device or a web server via the network N.

[0101] Next, the generation unit 21 analyzes the initial image Pf (which conceptually may also include a modified image Ps based on the initial image Pf) obtained by the reception unit 20, and generates information in natural language as suggested information regarding suitable modifications to the initial image Pf (S4). This information in natural language presents the impression after modification (candidate group of corrected images) when the initial image Pf is modified, and the intention behind the modification. In this embodiment, when generating the impression after modification, i.e., the information of the candidate group of corrected images, from the information in natural language, the generation unit 21 generates one or more groups containing two or more words as the candidate group of corrected images. The generation of this candidate group of corrected images is performed, for example, when the user half-presses the shooting button on the shooting device 200 to display a live view image of the desired subject.

[0102] In this case, the generation unit 21 generates multiple sets of correction image candidates, i.e., a group of correction image candidates, based on the caption obtained in S2 and the live view image corresponding to the initial image Pf. The group of correction image candidates consists of multiple keywords, i.e., captions, and can be described as a set of words that suggest the image after processing the changes to the live view image. Specific examples of the groups in the above-mentioned group of correction image candidates include: Gr. A: Noise added, Nostalgic, ..., Gr. B: CG style, Futuristic, ..., Gr. C: Horror, Dark, ...

[0103] Furthermore, the generation unit 21 may generate a group of images that show variations in brightness and color of the initial image Pf. In other words, specific parameters may be used as candidate corrected images. In this case, the generation unit 21 is equipped with AI that determines a group of specific parameters as preferred candidate corrected images for the initial image Pf by inputting at least one of the caption and the initial image Pf, or it may be available by calling it from an external system.

[0104] Furthermore, the generation unit 21 stores the information of the correction image candidate group generated as described above in the "candidate group" column of the input image table 201 for each corresponding image (S5).

[0105] Next, the determination unit 22 presents to the user, via the output unit 23, information on, for example, a group of correction image candidates generated by the generation unit 21 with respect to the initial image Pf (S6). For example, as shown in screen G1 of Figure 10, the determination unit 22 displays, for example, the information on the correction image candidate group, i.e., a list M1 of groups consisting of keyword groups (the "Gr." mentioned earlier), on the display of the shooting device 200 while the live view image is being displayed, in a selectable format.

[0106] In the example screen G1 in Figure 10, the list M1, which is a group of correction image candidates, lists three groups, "A" to "C," as correction image candidates. The keywords associated with the "image" in each group are "nostalgic" and "noise added" for group "A," "CG style" and "futuristic" for group "B," and "horror" and "dark" for group "C." A user viewing this screen G1 selects one or more images from list M1 that they consider to be suitable for their preferences or the initial image Pf.

[0107] Furthermore, in the example of screen G4 in Figure 13, the list M2, which is a group of correction image candidates, lists a total of four correction image candidates M21 ranging from "quite bright" to "quite dark". When a user views this screen G4, they select from list M2 the one they consider to have the brightness appropriate to their preference or the initial image Pf.

[0108] Such user selection actions include, but are not limited to, tapping the display area of ​​the target group (correction image candidate group) or target keyword (correction image candidate) on the touch panel of the shooting device 200. In the example of screen G3 in Figure 12, the user, who is the photographer, has selected a group for group A. The display area of ​​the selected group is controlled, for example, by inverting the color tone to gray or black. In the example of screen G4 in Figure 13, the user, who is the photographer, has selected a keyword for "quite bright". The display object for "quite bright" that was selected is controlled by inverting the color tone to black and the text color to white.

[0109] The determination unit 22 accepts the selection of any group from the list by the user, who is the photographer or image generator, as described above, and stores the information of the selected group, i.e., the group of correction image candidates, in the "Selected Group" column of the image table 222 (S7). The user may select multiple groups. Alternatively, the user may select a specific keyword (correction image candidate) from one group and a specific keyword (correction image candidate) from another group.

[0110] Next, the decision unit 22 identifies information for making corresponding changes based on the selected groups and keywords as described above, i.e., information for acquiring and generating the modified image Ps, from the setting item table 221 (see Figure 7) (S8). In this case, for example, with respect to the initial image Pf, which is a live view image, the camera parameters for shooting are identified to change its shooting conditions and convert the live view image into the modified image Ps. Alternatively, for example, the prompt Tp for obtaining the modified image Ps by regenerating the image using the image generation AI 211 is identified.

[0111] Among these, camera parameters may include parameters such as f-number, shutter speed, ISO sensitivity, color tone, and brightness. For example, when changing the initial image Pf to "futuristic," predefined values ​​such as f-number: 12, shutter speed: 1 / 1000 second, and ISO sensitivity: 100 are suitable. Also, prompt Tp will be predefined text, for example, when changing the initial image Pf to "nostalgic," such as "Set the background to a sunset sky and change the overall tone of the subject image to sepia." Alternatively, when changing the initial image Pf to "horror," predefined text such as "Set the time of day to night and change the overall tone of the subject image to dark gray or black. Blur the outline of the subject image to a certain extent." Or, when changing the initial image Pf to "quite bright," predefined text such as "Make the color tone of each subject, including the background, whitish" is suitable.

[0112] The determination unit 22 sets the camera parameter value from the setting item table 221 to the control unit in the shooting device 200 (S9). As a result, the output unit 23 acquires the modified image Ps as the result of shooting with the camera parameter when shooting operations are performed at subsequent timings, and displays it on the output device 15 (S10). The shooting device 200 can also change the live view image to the modified image Ps. The determination unit 22 also inputs the prompt Tp from the setting item table 221 to the image generation AI 211. As a result, the image generation AI 211 generates the modified image Ps, which is displayed on the output device 15 in the same manner as above.

[0113] In Figure 12, screen G3 shows the modified image Ps, which is the result of applying the "nostalgic" and "noise added" modifications to the initial image Pf shown in Figure 10. In Figures 13 (screen G4) and 14 (screen G5), the modified image Ps, which is brighter and generally whiter, is obtained after applying the "considerably brighter" modification to the initial image Pf. In Figures 13, screens G4 and G5 show that the brightness parameter is presented as a candidate for the corrected image in four stages, from "considerably brighter" to "considerably darker," and the user has selected "considerably brighter."

[0114] The determination unit 22 may also output the camera parameters and prompt Tp information, which are the above-mentioned set values, to the display of the shooting device 200 or user terminal 300 via the output unit 23. In the screen G6 shown in Figure 14, the parameter values ​​G61, such as the F value, are displayed for each group of correction image candidate group M1. In the screen G7 shown in Figure 15, the prompt Tp value G71 is displayed for each group of correction image candidate group M1. In the screen G8 shown in Figure 16, the exposure compensation value G81 is displayed for each of the correction image candidates M2. In the screen G9 shown in Figure 17, the prompt Tp value G91 is displayed for each of the correction image candidates M2.

[0115] When performing such output control, users familiar with camera operation and the use of the image generation AI 211 will be presented with information that they can directly understand regarding the modification of the initial image Pf. Alternatively, for users unfamiliar with camera operation and the use of the image generation AI 211, it will help them understand and learn the specific parameters that should be set in the shooting device 200 and the specific prompts Tp that should be input to the image generation AI 211 when performing shooting or image generation according to the "image" in the future.

[0116] [Modifications of this Embodiment] This embodiment is not limited to the above embodiment, and for example, the following modifications can be considered. These modifications will be described below. In the following, the differences between the modifications and the above embodiment will be explained.

[0117] (Regarding Modifications) As a modification of this embodiment, the determination unit 22 may accept a change operation for at least one of the keywords constituting the correction image candidate group when presenting the correction image candidate group to the user. In this case, as shown in screens G10 and G11 of Figure 18, for each row of each keyword M31 that constitutes the correction image candidate group M3, such as "building," "day and night," "dolphin," and "weather," a change operation selection object such as "change," "add," and "delete" is placed, and the user's selection of a keyword and instructions regarding the change operation are accepted.

[0118] In the example shown in screen G11, the keywords "day and night" are selected to "change" and "dolphin" is selected to "delete". As a result, the initial image Pf shows a school of dolphins in a river during the daytime, but in the modified image Ps, the time of day has changed to nighttime, and the dolphins that were swimming in the river have disappeared from the screen. In other words, the decision unit 22 can modify the user-specified keywords among the keywords that make up the group of correction image candidates as a modification operation.

[0119] These modifications can also be applied to each keyword (such as "nostalgic," "add noise," and "CG style" in the example in Figure 15) within a group of candidate images for correction (for example, Gr. A). In this case, the decision unit 22 might determine, for example, that in "Gr. A," selected by the user from the candidate image group for correction, the elements that the user should select, that is, the elements they want to adopt as elements of change, are "nostalgic" and "horror." In this case, the user inputs a modification instruction via the input device 14 to delete "add noise" and add "horror." The decision unit 22 receives this instruction regarding deletion and addition, and for example, deletes the value related to "add noise" and adds the value related to "horror" from the camera parameters or prompt Tp. This allows the modification trend of the live view image and generated image to be more to the user's liking.

[0120] Furthermore, the decision unit 22 may accept an operation to input information regarding the priority of the target keyword in the corrected image candidate group as the above modification operation. In this case, the decision unit 22 displays, for example, screen G12 in Figure 19 and accepts the user's selection operation via a checkbox-type interface G121 regarding the magnitude and absence of priority for each keyword in each group. Screen G12 shows that, among the keywords in group A, the priority of "nostalgic" is set to "high" and the priority of "noise addition" is set to "low". On the other hand, for groups B and C, no priority has been selected for any of the keywords, indicating that the groups themselves were not selected.

[0121] The decision unit 22 accepts changes to these priorities, for example, by increasing or decreasing values ​​that contribute to "nostalgia" among camera parameters and prompt Tp by a predetermined amount (e.g., 5% or one step) compared to the default values ​​in the setting item table 221. This allows the user to adjust the modification trends of live view images and generated images to better suit their preferences.

[0122] Furthermore, the decision unit 22 may accept an operation to input information regarding the priority among multiple correction image candidate groups as part of the above-mentioned modification operation. In this case, the decision unit 22 displays, for example, screen G12 in Figure 20 and accepts the user's selection operation regarding the magnitude and absence of priority for each group via a checkbox-type interface G131. Screen G13 shows that the priority of group A is "high", the priority of group B is "absent", and the priority of group C is "low". Note that the priority of group B is "absent", indicating that the group itself is not selected.

[0123] The decision unit 22 accepts modification operations regarding the priority between these groups, for example, it increases or decreases the values ​​of camera parameters and prompt Tp that contribute to "nostalgic, noise added" by a predetermined amount (e.g., 10% or 2 steps) compared to the default values ​​in the setting item table 221. It also increases or decreases the values ​​of camera parameters and prompt Tp that contribute to "horror, dark" by a predetermined amount (e.g., 5% or 1 step) compared to the default values ​​in the setting item table 221. By employing this control, the modification trends of live view images and generated images can be modified to better suit the user's preferences.

[0124] The determination unit 22 may also re-execute the acquisition and generation of the modified image Ps when, for example, the initial image Pf is modified, based on the corrected image candidate group that the user has re-selected from the multiple corrected image candidate groups or the corrected image candidate group that has been modified as described above. In this case, since the user is re-selecting from the multiple corrected image candidate groups, the determination unit 22 extracts the value in the "Candidate Group" column of the image table 222 and displays the information of each group (Gr. A to) on the display so that it can be selected.

[0125] Although specific embodiments of the present invention have been described above, these embodiments are merely examples given to facilitate understanding of the present invention and do not limit it. That is, the present invention can be modified or improved from the embodiments described below, without departing from its spirit. Furthermore, the present invention includes equivalents thereof. Moreover, embodiments of the present invention may include forms that combine the above embodiments with one or more of the following modifications.

[0126] (Regarding the computer constituting the information processing device) In the above embodiment, the information processing device of the present invention is configured by a computer used directly by the user, such as a user-owned PC (Personal Computer). However, it is not limited to this, and the information processing device of the present invention may be configured by a computer that can be used indirectly by the user, for example, the generation AI server 400. Here, the generation AI server 400 may be, for example, a server computer for cloud services, specifically a server computer for ASP (Application Service Provider), SaaS (Software as a Service), PaaS (Platform as a Service), or IaaS (Infrastructure as a Service). In this case, when the user inputs the necessary information on the user terminal 300 owned by the user, the generation AI server 400 performs various processes (calculations), including image generation, based on the input information, and the calculation results are output on the user terminal 300. In other words, the functions of the generation AI server 400, which constitutes the information processing device of the present invention, can be used on the user terminal 300. Alternatively, the information processing device may be configured using a PC or server computer, as well as a shooting device 200 or a smartphone (a type of user terminal 300) used by the user.

[0127] (Regarding the processing) In this embodiment, each process is executed on any computer. Furthermore, any computer may execute these processes using a processor as hardware, a program as software, or a combination thereof. In that case, the processor is configured to work in cooperation with the program to execute the various processes in this embodiment, and can function as a unit or means in this embodiment. Also, the execution order of the processes by the processor is not limited to the order described and may be changed as appropriate. Any computer may be a general-purpose computer, a computer designed for a specific purpose, a workstation, or any other system capable of executing each process.

[0128] (Regarding the Processor) The processor in this embodiment may be composed of one or more hardware components, and the type of hardware is not limited. For example, the processor may be composed of programmable logic devices such as a CPU (Central Processing Unit), MPU (Micro Processing Unit), FPGA (Field Programmable Gate Array), dedicated circuits for executing specific processing such as an ASIC (Application Specific Integrated Circuit), GPU (Graphic Processing Unit), or NPU (Neural Processing Unit). The processor also has various parts (Units) or means (Means) that execute the various processing components in this embodiment. Alternatively, the type of hardware may be a combination of different types of hardware. When multiple hardware components are configured to execute one or more processes of a processor, these components may reside in physically separate devices or in the same device. Furthermore, in any embodiment, the order of the processes performed by the processor is not limited to the order described above and may be changed as appropriate. The hardware components are composed of electrical circuits (circuits) and the like, which are combinations of circuit elements such as semiconductor elements.

[0129] (Regarding the configuration) Furthermore, the various configurations in this embodiment may be realized by hardware, software, firmware, microcode, or a combination thereof. Software, firmware, and microcode are composed of programs. The program may also be, for example, a group of program modules, and each of its functions may be realized by a processor configured to execute its respective function. Furthermore, the program may be firmware or software such as microcode. The program may also be, for example, a group of program modules, and each of its functions may be realized by a processor configured to execute its respective function. The program may be program code or multiple code segments stored in one or more non-temporary computer-readable media (e.g., storage media or other storage). The program may be divided and stored in multiple non-temporary computer-readable media located in devices that are physically separated from each other. Program code or code segments may represent any combination of procedures, functions, subprograms, routines, subroutines, modules, software packages, classes, or instructions, data structures, or program statements. Program code or code segments may be connected to other code segments or hardware circuits by sending and receiving information, data, arguments, parameters, or memory contents.

[0130] N Network 5 Information Processing System 100 Information Processing Device 11 Processor 12 Auxiliary Storage Device 121 Program 13 Main Storage Device 14 Input Device 15 Output Device 16 Communication Device 20 Reception Unit 201 Input Image Table 202 Text Information Table 21 Generation Unit 211 Image Generation AI 212 Caption Generation AI 213 Image Definition Table 22 Decision Unit 221 Setting Item Table 222 Image Table 23 Output Unit 200 Shooting Device 300 User Terminal 400 Generation AI Server Tp Prompt Pf Initial Image Ps Modified Image M1-M3 Modified Image Candidate Group (Information in Natural Language)

Claims

1. An information processing device comprising a processor that makes suggestions regarding changes to an image, wherein the processor performs the following: a process of receiving an image as input; a process of analyzing the image and generating information in natural language as information regarding changes to the image; and a process of outputting the information in natural language.

2. The information processing apparatus according to claim 1, wherein the processor generates information corresponding to the modified image when the image is modified, as natural language information.

3. The information processing apparatus according to claim 1, wherein the information in natural language is a natural language expression that includes the intention to change the image.

4. The information processing apparatus according to claim 1, wherein the information in natural language includes the intention to change the image and is expressed in natural language that is understandable to the user.

5. The information processing apparatus according to claim 2, wherein the processor generates one or more groups containing two or more words as candidates for the modified image, as information relating to the modification.

6. The information processing apparatus according to claim 1, wherein the processor outputs information regarding settings for acquiring a modified image when a change is made to the image, based on the information in natural language.

7. The information processing apparatus according to claim 6, wherein the processor outputs information of a plurality of groups which are candidates for the modified image, accepts a user's selection of which group to be used as the modified image, and configures the image capture device based on the words of the group.

8. The information processing apparatus according to claim 1, wherein the processor outputs information regarding settings for making changes to the image and generating a modified image based on the natural language information.

9. The information processing apparatus according to claim 5, wherein the processor outputs information of a plurality of groups which are candidates for the modified image, accepts a user's selection of which group to be the modified image, and generates the modified image using a generation AI based on the words of the group.

10. The information processing apparatus according to claim 1, wherein the processor analyzes the image to extract words that represent the image and generates information about the change based on those words.

11. The information processing apparatus according to claim 1, wherein the processor accepts user input on the outputted natural language information.

12. The information processing apparatus according to claim 5, wherein the processor accepts a modification operation for at least one of the two or more words included in the group.

13. The information processing apparatus according to claim 12, wherein the modification operation is an operation to input information regarding the priority of two or more words included in the group.

14. The information processing apparatus according to claim 12, wherein the modification operation is an operation to input information to modify two or more words included in the group.

15. The information processing apparatus according to claim 11, wherein the processor receives information for two or more words that are included in two or more groups from the outputted natural language information as a user modification operation.

16. The information processing apparatus according to claim 1, wherein the processor stores the generated natural language information in association with the image.

17. The information processing apparatus according to claim 16, wherein the processor regenerates the modified image when a change is made to the image, based on any of the natural language information stored with respect to the image.

18. The information processing apparatus according to claim 1, wherein the processor determines, based on the information in natural language, to change or add at least one of the color, brightness, and subject of the image as the content of the image change.

19. An information processing method comprising: a process of receiving an image input by a processor; a process of analyzing the image by a processor and generating information in natural language as information relating to changes in the image; and a process of outputting the information in natural language by a processor.

20. A program for causing a computer to perform each step included in the information processing method described in claim 19.

21. A computer-readable recording medium on which a program is recorded causing a computer to perform each step included in the information processing method described in claim 19.

Images