Information processing device, information processing method, and information processing program

Abstract

To provide an information processing device, an information processing method, and an information processing program that allow generation of an appropriate text related to an image.SOLUTION: An information processing device 10 includes at least one processor. The processor obtains an image and related object information indicating each of a plurality of related objects related to the image, and generates a text based on a relationship of at least two different ones of the related objects, based on the image and the related object information.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The technology disclosed herein relates to an information processing device, an information processing method, and an information processing program. [Background technology]

[0002] 2. Description of the Related Art Conventionally, there is known a technique for supporting the creation of a photo book using images owned by a user.

[0003] For example, Patent Document 1 discloses a method for determining a title of a photo book by analyzing image data used in the photo book, determining a design template based on the title, and allocating the image data and the title using the design template. For example, Patent Document 2 discloses a method for generating photo book data using images determined to be suitable for use in a photo book among image data received from a user terminal when a photo book creation instruction is received by voice from a user. For example, Patent Document 3 discloses a method for assisting the user with inputting text by generating text describing the image and notifying the user. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2019-149020 A [Patent Document 2] JP 2019-160185 A [Patent Document 3] JP 2020-052947 A Summary of the Invention

[0005] One embodiment of the technology disclosed herein provides an information processing device, an information processing method, and an information processing program capable of generating appropriate text related to an image. [Means for solving the problem]

[0006] A first aspect of the present disclosure is an information processing device comprising at least one processor, which acquires an image and related object information representing each of a plurality of related objects associated with the image, and generates text based on a relationship between at least two different related objects based on the image and the related object information.

[0007] In the first aspect, the processor may generate text when at least one of the multiple related objects is set as a source and at least another related object is set as a destination.

[0008] In the first aspect, the related object may be at least one subject included in the image, or the owner, creator, photographer, or viewer of the image.

[0009] In the first aspect, the subject may be a person, an animal, or an object.

[0010] In the first aspect, the processor may generate the text by referring to a template of text that is predetermined according to a combination of at least two different associated entities.

[0011] In the above first aspect, the processor may generate text using a pre-trained model that takes image and related object information as input and outputs text.

[0012] In the first aspect, the processor may receive designation of at least two related entities among the plurality of related entities, and generate text according to the relationship between the two designated related entities.

[0013] In the first aspect, the processor may generate text for each combination of at least two different associated entities, and accept a selection of at least one of the generated text for each combination of associated entities.

[0014] In the first aspect above, the processor may generate the related object information based on at least one subject included in the image.

[0015] In the first aspect, the processor may receive an input of related entity information.

[0016] In the first aspect, the processor may obtain attendant information of the image, and generate related object information based on the attendant information.

[0017] In the first aspect, the processor may obtain additional information about the image, and generate text based on the image, the related object information, and the additional information.

[0018] In the first aspect, the processor may obtain a plurality of images relating to the same associated entity, and generate text according to a comparison of the plurality of images.

[0019] A second aspect of the present disclosure is an information processing method in which a computer executes a process of acquiring an image and related object information representing each of a plurality of related objects associated with the image, and generating text based on a relationship between at least two different related objects based on the image and the related object information.

[0020] A third aspect of the present disclosure is an information processing program for causing a computer to execute a process of acquiring an image and related object information representing each of a plurality of related objects associated with the image, and generating text based on the relationship between at least two different related objects based on the image and the related object information. [Brief description of the drawings]

[0021] [Figure 1] FIG. 1 is a schematic diagram illustrating an example of a configuration of an information processing system. [Diagram 2] FIG. 1 illustrates an example of a photobook including images and text. [Diagram 3] FIG. 2 is a block diagram showing an example of a hardware configuration of an information processing device. [Figure 4] FIG. 2 is a block diagram showing an example of a functional configuration of an information processing device. [Diagram 5] FIG. 13 is a diagram showing an example of a text template. [Figure 6] FIG. 1 illustrates an example of a photobook including images and text. [Figure 7] FIG. 13 is a diagram showing an example of a screen for selecting text. [Figure 8] FIG. 13 is a diagram showing an example of a screen for generating text. [Figure 9] 11 is a flowchart illustrating an example of a flow of information processing. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings.

[0023] First, an information processing system 100 to which an information processing device 10 according to the present embodiment is applied will be described with reference to Fig. 1. Fig. 1 is a diagram showing a schematic configuration of the information processing system 100. As shown in Fig. 1, the information processing system 100 includes the information processing device 10 and a plurality of terminal devices 14A to 14C owned by a plurality of users 18A to 18C, respectively. Hereinafter, when there is no need to distinguish between the users 18A to 18C, they will be simply referred to as "users 18", and when there is no need to distinguish between the terminal devices 14A to 14C, they will be simply referred to as "terminal devices 14".

[0024] The terminal device 14 is a device having a camera function, an image playback and display function, an image transmission and reception function, etc. The camera function is a function having an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor, and obtaining an image 52 (see FIG. 2) of a subject by focusing the subject light captured from a lens on the imaging element. For example, a smartphone, a tablet terminal, a personal computer, a digital camera, etc. can be appropriately applied as the terminal device 14. Note that the terminal device 14 is not limited to being composed of one device, and may be composed of multiple devices, such as a combination of a digital camera and a personal computer.

[0025] The terminal device 14 is connected to the information processing device 10 via a wired or wireless network 12 so as to be able to communicate with each other. The network 12 is, for example, a wide area network (WAN) such as the Internet or a public communication network. The terminal device 14 transmits (uploads) images 52 to the information processing device 10. In addition, the terminal device 14 receives (downloads) a photo book 50 from the information processing device 10.

[0026] The information processing device 10 creates a photo book 50 using images 52 acquired by the terminal device 14, and transmits the created photo book 50 to the terminal device 14. Specifically, the information processing device 10 creates text 54 appropriate for the images 52 received from the terminal device 14, and attaches the text 54 to the corresponding images 52, thereby creating the photo book 50.

[0027] A photo book 50 is a collection of multiple images 52 based on a theme, such as various events such as school events, trips, and weddings, as well as growth records. In addition, the images 52 may be accompanied by text 54 about the subject of the image 52, the photographer, the date and time of the photo, the location and circumstances of the photo, etc.

[0028] FIG. 2 shows an example of a photo book 50. In the example of FIG. 2, the photo book 50, entitled "Taro's Growth Record," includes two images 52X and 52Y, and texts 54X and 54Y are attached to the images 52X and 52Y, respectively. The images 52X and 52Y include a user 18A and a child 19A as subjects, respectively. In this example, the child 19A is a child named "Taro," and the user 18A is Taro's father. The user 18B, who photographed the images 52X and 52Y, is Taro's mother, and the user 18C, who is a viewer of the photo book 50, is Taro's grandparent.

[0029] The user 18 takes images 52 using the camera function of the terminal device 14, and views the photo book 50 using the image playback display function. According to such an information processing system 100, the user 18A who is the subject of the images 52, the user 18B who is the photographer, and the user 18C who is the viewer can view the photo book 50 at their own convenience. In addition, the images taken by each of the users 18A to 18C can be compiled into one photo book 50, and each of the users 18A to 18C can edit one photo book 50.

[0030] Incidentally, the text 54 attached to the image 52 in the photo book 50 may be a message such as a request or instruction from a related object of the image 52 to another related object. For example, the text 54X shown in Fig. 2, "I'm happy that Daddy held me," is intended as a message from the child 19A (Taro) who is the subject, to the user 18A (father), who is another subject. Also, the text 54Y shown in Fig. 2, "Taro, you've grown so much! Mommy can't hold you anymore," is intended as a message from the user 18B (mother) who is the photographer, to the child 19A (Taro), who is the subject.

[0031] There are cases where the image 52 can be made more impressive by adding text 54 addressed from one related entity to another related entity, such as text 54X and 54Y, to the image 52. Therefore, the information processing device 10 according to the present embodiment generates text 54 based on the relationship between at least two different related entities related to the image 52. Details of the information processing device 10 will be described below.

[0032] First, an example of a hardware configuration of the information processing device 10 will be described with reference to Fig. 3. As shown in Fig. 3, the information processing device 10 includes a CPU (Central Processing Unit) 21, a non-volatile storage unit 22, and a memory 23 as a temporary storage area. The information processing device 10 also includes a display 24 such as a liquid crystal display, an operation unit 25 such as a touch panel, a keyboard, and a mouse, and an I / F (Interface) unit 26. The I / F unit 26 performs wired or wireless communication with the terminal device 14 and other external devices. The CPU 21, the storage unit 22, the memory 23, the display 24, the operation unit 25, and the I / F unit 26 are connected to each other via a bus 28 such as a system bus and a control bus so that various information can be exchanged between them.

[0033] The storage unit 22 is realized by a storage medium such as a hard disk drive (HDD), a solid state drive (SSD), and a flash memory. An information processing program 27 for the information processing device 10 is stored in the storage unit 22. The CPU 21 reads the information processing program 27 from the storage unit 22, expands it in the memory 23, and executes the expanded information processing program 27. The CPU 21 is an example of a processor of the present disclosure. For example, a server computer, a personal computer, a smartphone, a tablet terminal, a wearable terminal, or the like can be appropriately applied as the information processing device 10.

[0034] Next, an example of a functional configuration of the information processing device 10 will be described with reference to Fig. 4. As shown in Fig. 4, the information processing device 10 includes an acquisition unit 30, a generation unit 32, and a control unit 34. The CPU 21 executes the information processing program 27, causing the CPU 21 to function as each of the functional units of the acquisition unit 30, the generation unit 32, and the control unit 34.

[0035] The acquiring unit 30 acquires at least one image 52 from the terminal device 14. The image 52 is, for example, an image (i.e., a photo) captured using a camera function of the terminal device 14. The image 52 may be provided with additional information such as the date and time of capture, the location, the direction, the ID of the photographer, and the ID of the capture device. An example of the format of the additional information is EXIF ​​(Exchangeable image file format).

[0036] The acquiring unit 30 also acquires related object information indicating each of a plurality of related objects related to the acquired image 52. The related object is, for example, at least one subject included in the image 52, or the owner, creator, photographer, or viewer of the image 52. In other words, the related object information is information that identifies the subject included in the image 52, or the owner, creator, photographer, or viewer of the image 52.

[0037] Specifically, the acquiring unit 30 may generate related object information based on a subject included in the acquired image 52. For example, the acquiring unit 30 may analyze the image 52 to identify a subject appearing in the image 52, and generate related object information representing the identified subject. As a method for analyzing the image 52, for example, a known method such as a method using a cascade classifier or a method using pattern matching can be appropriately applied.

[0038] The acquisition unit 30 may also accept input of related object information by the user 18. For example, the acquisition unit 30 may control the display 24 or the display of the terminal device 14 to display a screen for inquiring about related objects of the image 52, and acquire information input on the screen as related object information.

[0039] The acquiring unit 30 may also acquire the incidental information of the image 52 and generate related object information based on the incidental information. As described above, the incidental information of the image 52 may include the identification ID of the photographer. In this case, the acquiring unit 30 may specify, as the related object information, information indicating the photographer specified based on the identification ID of the photographer included in the incidental information.

[0040] Furthermore, the acquisition unit 30 may acquire information about the user 18 who is the owner of the terminal device 14 from the terminal device 14, and generate related object information based on the information about the user 18. The user 18 may correspond to at least one of the owner, creator, photographer, and viewer of the image 52. The information about the user 18 is, for example, information indicating the identification ID, name, age, sex, family structure, etc. of the user 18 that are preset in the terminal device 14.

[0041] The subject as a related object represented by the related object information is not limited to a person, but may be an animal or an object. The object is various man-made objects and natural objects, and examples of the object include tools, toys, machines, buildings, plants, and landforms such as mountains, seas, and rivers.

[0042] The related object represented by the related object information may be an individual or a collection. For example, the related object information may represent a collection of multiple people, animals, or things, such as "parents and children," "grandparents," "all students at a school," "teenage girls," and "three dogs."

[0043] In addition, the degree of specificity of the related object represented by the related object information is not particularly limited, and information indicating the attributes of the related object and information identifying an individual can be appropriately applied. For example, even information representing the same related object can be expressed differently, such as "animal," "dog," "pet," and "Wan-kun" (nickname for a pet dog), and any of these may be used as the related object information of the present disclosure.

[0044] The generating unit 32 generates text 54 based on the relationship between at least two different related objects, based on the image 52 and related object information acquired by the acquiring unit 30. Specifically, the generating unit 32 generates text 54 in a case where at least one of a plurality of related objects related to the image 52 is set as the source and at least another related object is set as the destination. In other words, the text 54 is a communication sentence such as a thought, request, instruction, etc., from one related object of the image 52 to another related object.

[0045] Moreover, the text 54 is generated according to the contents of the image 52. For example, the generation unit 32 may analyze the subject and its situation included in the image 52, and generate the text 54 according to the analysis result. As a method for analyzing the image 52, for example, a known method such as a method using a cascade classifier or a method using pattern matching can be appropriately applied.

[0046] In the example of Fig. 2, it is assumed that the acquisition unit 30 acquires related object information representing each of the child 19A (Taro) and user 18A (father) who are the subjects, the user 18B (mother) who is the photographer, and the user 18C (grandparents) who is the viewer as related objects of the image 52X. It is also assumed that the generation unit 32 analyzes the situation of the image 52X, such as "the father is holding Taro", "Taro is smiling", and "the father is smiling". The generation unit 32 generates the text 54X, "I'm happy that daddy is holding me", assuming the thoughts of the child 19A (Taro) to the user 18A (father) under these circumstances.

[0047] Note that multiple related entities may be set as the source and destination of the generated text 54. For example, text 54 may be generated with multiple people set as destinations, such as "Both Dad and Taro look happy." Although such text 54 may appear to be a simple explanation or description of image 52, it represents the impression of the subject seen by the viewer, that is, it is text 54 addressed to the subject by the viewer, and is included in an example of text in the present disclosure.

[0048] A known method can be appropriately applied as a method for generating the text 54. For example, the generation unit 32 may generate the text 54 by referring to a template of the text 54 that is predetermined according to a combination of at least two different related objects. FIG. 5 shows an example of the template. FIG. 5 shows a template of the text 54 that can be added to an image 52 in a situation where a child and a father are subjects, a mother is a photographer, a grandparent is a viewer, and the father is holding a child. Such templates may be prepared in advance for each related object and each situation, and stored in the storage unit 22, an external database server, or the like.

[0049] The generation unit 32 may acquire text 54 to be added to the image 52 by referring to an appropriate template based on the analysis result of the image 52 and the related object information. Also, the generation unit 32 may generate more appropriate text 54 by modifying at least a part of the text 54 acquired from the template. For example, the generation unit 32 may replace the portion "(child's name)" in the template of FIG. 5 with "Taro" based on the related object information.

[0050] Furthermore, for example, the generation unit 32 may generate the text 54 using a trained model that has been trained in advance to receive an image and related object information as input and to output text. This trained model is, for example, a machine learning model trained using training data including a combination of an image, related object information of the image, and text to be added to the image. As such a machine learning model, for example, a known model such as a convolutional neural network (CNN) and a recurrent neural network (RNN) can be appropriately applied.

[0051] In addition, when there are multiple images 52 related to the same related object, text 54 to be added to one image may be generated according to the content of another image. For example, the generation unit 32 may obtain multiple images 52 related to the same related object, and generate text 54 according to a comparison of the multiple images 52. The text 54 according to the comparison may represent, for example, a change over time in the appearance and form of the subject. In the example of FIG. 2, the generation unit 32 compares images 52X and 52Y, each of which includes a child 19A (Taro) as a subject, to generate text 54Y including the sentence "Taro, you've grown so big!"

[0052] Furthermore, the generation unit 32 may generate text 54 based on the image 52, related object information, and additional information. As described above, the additional information of the image 52 may include information such as the shooting date and time, the location, and the orientation. The generation unit 32 may generate text 54 such as "It was cold at the amusement park in December, but it was fun" using the shooting date and time and location information attached to the image 52.

[0053] As described above, an animal or an object may be applied as the related object. For example, the generation unit 32 may generate text 54 expressing the thoughts of an animal or an object when the animal or object is anthropomorphized as the source. Also, for example, the generation unit 32 may generate text 54 expressing the thoughts of the owner, creator, photographer, or viewer of the image 52 when the animal or object is set as the destination.

[0054] FIG. 6 shows an example of a photo book 50P including texts 54P-54Q in the case where at least one related object is an animal or thing. In FIG. 6, the text 54P attached to the image 52P is assumed to be a message from the subject dog 19B to another subject child 19A (Taro). The text 54Q attached to the image 52Q is assumed to be a message from the subject flying disc 19C to another subject dog 19B, and is generated according to the content of another image 52P, "Dog 19B playing with flying disc 19C". In this example, the dog 19B is given the nickname "Wan-kun". The text 54R attached to the image 52R is assumed to be a message from the photographer (not shown) to the subject mountain.

[0055] Furthermore, the generation unit 32 may generate text 54 for each combination of at least two different related objects for the image 52, and may accept selection of at least one of the generated texts 54 for each combination of related objects. That is, the generation unit 32 may generate a plurality of candidates for the text 54 to be added to the image 52 with different senders and destinations, and allow the user 18 to select at least one of them. Note that the generation unit 32 does not need to generate text 54 for all combinations of related objects for the image 52, and may generate candidates for the text 54 for at least some of the combinations.

[0056] FIG. 7 shows an example of a screen 80 for selecting text 54P to be added to image 52P. Screen 80 includes text candidates 541 to 544 generated for each of four combinations of related items of image 52P. Text candidate 541 is text for a combination of a sender being a photographer and a destination being a subject child 19A (Taro) and a dog 19B (Wan-kun). Text candidate 542 is text for a combination of a sender being a viewer and a destination being a subject dog 19B (Wan-kun). Text candidate 543 is text for a combination of a sender being a subject child 19A (Taro) and a destination being a subject dog 19B (Wan-kun). Text candidate 544 is text for a combination of a sender being a subject dog 19B (Wan-kun) and a destination being a subject child 19A (Taro).

[0057] The screen 80 also includes a check box 82 for selecting at least one of the text candidates 541-544. The control unit 34 controls the screen 80 to be displayed on the display 24 or the display of the terminal device 14. The user 18 checks the screen 80 and selects, by using the check box 82, at least one of the text candidates 541-544 that the user 18 desires to add to the image 52P. The generation unit 32 determines the at least one selected of the text candidates 541-544 as the text 54P to be added to the image 52P. FIG. 7 illustrates a case where the text candidate 544 is selected.

[0058] Furthermore, the generation unit 32 may receive designation of at least two related objects among a plurality of related objects for the image 52, and generate the text 54 according to the relationship between the two designated related objects. That is, the generation unit 32 may have the user 18 designate the source and destination of the text 54 before generating the text 54.

[0059] 8 shows an example of a screen 90 for specifying related objects used in generating text 54. Screen 90 includes image 52P and buttons 92 indicating the photographer and viewer as related objects not included in image 52P. The buttons 92 indicating the photographer and viewer include buttons for specifying specific attributes ("Father," "Mother," and "Friend") and buttons for not specifying attributes ("Photographer" and "Viewer"). Image 52P includes, as subjects, child 19A (Taro), dog 19B (Wan-kun), flying disc 19C, and mountain 19D.

[0060] The control unit 34 controls the display 24 or the display of the terminal device 14 to display the screen 90. The user 18 checks the screen 90 and specifies the source and destination of the text 54 to be added to the image 52P. The generation unit 32 generates the text 54 according to the source and destination specified by the user 18.

[0061] For example, the user 18 performs a drag-and-drop operation with the mouse pointer 94 from the button 92 or related object in the image 52P that the user 18 desires to set as the source, to the button 92 or related object in the image 52P that the user desires to set as the destination. Fig. 8 shows an example of text 54P that is generated when the "Photographer" button 92 is specified as the source, and "Child 19A (Taro)" in the image 52P is specified as the destination.

[0062] The control unit 34 associates the text 54 generated by the generation unit 32 with the images 52, and stores the resulting photo book 50 in the storage unit 22, an external database server, or the like. Furthermore, when a request to view the photo book 50 is received from the terminal device 14, the control unit 34 transmits the photo book 50 to the terminal device 14.

[0063] Next, the operation of the information processing device 10 according to the present embodiment will be described with reference to Fig. 9. In the information processing device 10, the CPU 21 executes the information processing program 27, thereby executing the information processing shown in Fig. 9. The information processing is executed, for example, when a command to start execution is given by the user 18.

[0064] In step S10, the acquisition unit 30 acquires at least one image 52 from the terminal device 14. In step S12, the acquisition unit 30 acquires related entity information representing each of a plurality of related entities related to the image 52 acquired in step S10. In step S14, the generation unit 32 generates text 54 based on the relationship between at least two different related entities, based on the image 52 acquired in step S10 and the related entity information acquired in step S12. When step S12 is completed, this information processing ends.

[0065] As described above, an information processing device 10 according to one embodiment of the present disclosure includes at least one processor, which acquires an image 52 and related object information representing each of a plurality of related objects associated with the image 52, and generates text 54 based on the relationship between at least two different related objects based on the image 52 and the related object information.

[0066] That is, the information processing device 10 can generate appropriate text 54 related to the image, such as a message such as a thought, request, or instruction from a related entity to another related entity of the image 52. Therefore, such text 54 can be added to the image 52, making the image 52 more impressive.

[0067] In the above embodiment, the technology of the present disclosure is described as being applied to the creation of a photo book 50, but the present disclosure is not limited to this. The technology of the present disclosure can be applied to the creation of various items that include a combination of images 52 and text 54, and may be applied to the creation of, for example, albums, photo books, photo ledgers, postcards, message cards, etc. Furthermore, the means for realizing these is not limited to electronic data, and may be, for example, printed on a medium such as paper or resin film.

[0068] In the above embodiment, the image 52 is a photograph taken using the camera function of the terminal device 14, but the present invention is not limited to this. For example, image data generated by CG (Computer Graphics) may be used as the image 52. For example, a painting may be used as the subject, and in this case, the image 52 of the painting may be obtained by reading the paper on which the painting is drawn using a scanner. When the image 52 includes CG or a painting, the related object information may represent a person, animal, or thing depicted by the CG or painting. For example, when the image 52 is a frame of an animation work, the acquisition unit 30 may acquire related object information representing a character in the animation work, and the generation unit 32 may generate text 54 that assumes the feelings of one character toward another character.

[0069] In each of the above embodiments, the following various processors can be used as the hardware structure of the processing unit that executes various processes, such as the acquisition unit 30, the generation unit 32, and the control unit 34. As described above, the above various processors include a CPU, which is a general-purpose processor that executes software (programs) and functions as various processing units, as well as a programmable logic device (PLD), which is a processor whose circuit configuration can be changed after manufacture, such as an FPGA (Field Programmable Gate Array), a dedicated electric circuit, which is a processor having a circuit configuration designed specifically for executing specific processes, such as an ASIC (Application Specific Integrated Circuit), etc.

[0070] A single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, multiple processing units may be configured with a single processor.

[0071] As an example of configuring multiple processing units with one processor, first, there is a form in which one processor is configured with a combination of one or more CPUs and software, as typified by computers such as client and server, and this processor functions as multiple processing units. Second, there is a form in which a processor is used that realizes the functions of the entire system including multiple processing units with one IC (Integrated Circuit) chip, as typified by System on Chip (SoC), etc. In this way, the various processing units are configured using one or more of the above various processors as a hardware structure.

[0072] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements.

[0073] In the above embodiment, the information processing program 27 in the information processing device 10 is stored in advance in the storage unit 22, but the present invention is not limited to this. The information processing program 27 may be provided in a form recorded on a recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), or a USB (Universal Serial Bus) memory. The information processing program 27 may also be downloaded from an external device via a network. Furthermore, the technology of the present disclosure extends to a storage medium that non-temporarily stores a program, in addition to the program.

[0074] The technology of the present disclosure can also be appropriately combined with the above-mentioned embodiment examples and examples. The above-mentioned description and illustrated contents are detailed descriptions of the parts related to the technology of the present disclosure, and are merely one example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the description and illustrated contents shown above, within the scope of the gist of the technology of the present disclosure.

[0075] The following supplementary notes are further disclosed regarding the above embodiment. [Appendix 1] at least one processor; The processor, Obtaining an image and related object information representing each of a plurality of related objects related to the image; Generate text based on the relationship between at least two different related entities based on the image and the related entity information. Information processing device. [Appendix 2] The processor, generating said text when at least one of said plurality of related objects is set as a source and at least one other of said plurality of related objects is set as a destination; 2. An information processing device according to claim 1. [Appendix 3] The related entity is at least one subject included in the image, or the owner, creator, photographer, or viewer of the image. 3. An information processing device according to claim 1 or 2. [Appendix 4] The subject is a person, an animal, or an object. 4. The information processing device according to claim 3. [Appendix 5] The processor, generating said text by referencing a template of text that is predefined in response to at least two different combinations of said associations; 5. An information processing device according to any one of claims 1 to 4. [Appendix 6] The processor, The text is generated using a pre-trained model that is pre-trained to use the image and the related object information as input and output the text. An information processing device according to any one of Supplementary Note 1 to Supplementary Note 5. [Appendix 7] The processor, Accepting designation of at least two of the plurality of related entities; Generate the text according to the relationship between the two specified related objects. 7. An information processing device according to any one of claims 1 to 6. [Appendix 8] The processor, generating said text for each combination of at least two different said associations; Accepting a selection of at least one of the texts for each of the generated associations. 8. An information processing device according to any one of claims 1 to 7. [Appendix 9] The processor, The related object information is generated based on at least one object included in the image. An information processing device according to any one of Supplementary Note 1 to Supplementary Note 8. [Appendix 10] The processor, Accept the input of the related object information 10. An information processing device according to any one of claims 1 to 9. [Appendix 11] The processor, Acquire supplementary information of the image; The related object information is generated based on the supplementary information. An information processing device according to any one of Supplementary Note 1 to Supplementary Note 10. [Appendix 12] The processor, Acquire supplementary information of the image; The text is generated based on the image, the related object information, and the additional information. An information processing device according to any one of Supplementary Note 1 to Supplementary Note 11. [Appendix 13] The processor, Obtaining multiple images related to the same related object; generating said text responsive to a comparison of said plurality of images 13. An information processing device according to any one of claims 1 to 12. [Appendix 14] Obtaining an image and related object information representing each of a plurality of related objects related to the image; Generate text based on the relationship between at least two different related entities based on the image and the related entity information. An information processing method in which processing is performed by a computer. [Appendix 15] Obtaining an image and related object information representing each of a plurality of related objects related to the image; Generate text based on the relationship between at least two different related entities based on the image and the related entity information. An information processing program that enables a computer to execute processing. [Explanation of symbols]

[0076] 10. Information processing device 12 Network 14A~14C Terminal equipment 18A~18C Users 19A Children 19B Dog 19C Flying Disc 19D Mountain 21 CPU 22 Memory section 23 Memory 24 Display 25 Input section 26 Network Interface 27 Information Processing Programs 28 Bus 30 Acquisition Department 32 Generation part 34 Control section 50, 50P Photo Book 52, 52X, 52Y, 52P~52R Images 54, 54X, 54Y, 54P~54R Text 80, 90 screens 82 Checkbox 92 Buttons 94 Mouse Pointer 100 Information Processing Systems 541~544 Text candidates

Claims

1. At least one processor; The processor, Obtaining an image and related object information representing each of a plurality of related objects related to the image; Generate text based on the relationship between at least two different related entities based on the image and the related entity information. Information processing device.

2. The processor, generating said text when at least one of said plurality of related objects is set as a source and at least one other of said related objects is set as a destination; The information processing device according to claim 1 .

3. The related entity is at least one subject included in the image, or the owner, creator, photographer, or viewer of the image. The information processing device according to claim 1 .

4. The subject is a person, an animal, or an object. The information processing device according to claim 3 .

5. The processor, generating said text by referencing a template of text that is predefined in response to a combination of at least two different said associations; The information processing device according to claim 1 .

6. The processor, The text is generated using a pre-trained model that is pre-trained to use the image and the related object information as input and output the text. The information processing device according to claim 1 .

7. The processor, Accepting designation of at least two of the plurality of related entities; Generate the text according to the relationship between the two specified related objects. The information processing device according to claim 1 .

8. The processor, generating said text for each combination of at least two different said associations; Accepting a selection of at least one of the texts for each of the generated associations. The information processing device according to claim 1 .

9. The processor, The related object information is generated based on at least one object included in the image. The information processing device according to claim 1 .

10. The processor, Accept the input of the related object information The information processing device according to claim 1 .

11. The processor, Acquire supplementary information of the image; The related object information is generated based on the supplementary information. The information processing device according to claim 1 .

12. The processor, Acquire supplementary information of the image; The text is generated based on the image, the related object information, and the additional information. The information processing device according to claim 1 .

13. The processor, Obtaining multiple images related to the same related object; generating said text responsive to a comparison of said plurality of images The information processing device according to claim 1 .

14. Obtaining an image and related object information representing each of a plurality of related objects related to the image; Generate text based on the relationship between at least two different related entities based on the image and the related entity information. An information processing method in which processing is performed by a computer.

15. Obtaining an image and related object information representing each of a plurality of related objects related to the image; Generate text based on the relationship between at least two different related entities based on the image and the related entity information. An information processing program that enables a computer to execute processing.

Images