Information processing system, program and information processing method

Abstract

To provide an information processing system, etc. allowing a user to easily acquire information about an image group used for machine learning.SOLUTION: There is provided an information processing system including a processor. The information processing system causes the processor to execute: an information acquisition step of acquiring related information related to an image group used for machine learning; and an output step of outputting metadata representing the related information acquired for the image group in association with an image generated by an artificial intelligence module subjected to machine-learning using the image group.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an information processing system, a program, and an information processing method.

Background Art

[0002] Patent Document 1 discloses a learning process in which a combination of each image in a sub-learning image group, which is a group of a plurality of images in which a subject is photographed, and the pose information of the specific subject in each image in the sub-learning image group is used as teacher data, and a technique for newly generating an image of the specific subject based on the parameters of an image generator obtained by performing the learning process.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In order for an artificial intelligence module to generate an image, machine learning using existing images as teacher data is performed. At that time, grasping the images used in the machine learning may be important when using the generated images.

[0005] In view of the above circumstances, the present invention aims to provide an information processing system or the like that can easily grasp information about an image group used in machine learning.

Means for Solving the Problems

[0006] According to one aspect of the present invention, an information processing system including a processor is provided. In this information processing system, in the information acquisition step, the processor acquires related information related to an image group used for machine learning. In the output step, metadata indicating the related information acquired for the image group is output in association with an image generated by an artificial intelligence module that has performed machine learning using the image group.

[0007] According to such an aspect, it is possible to easily grasp information about the image group used for machine learning.

Brief Description of Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Embodiments for Carrying Out the Invention

[0009] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Various characteristic matters shown in the following embodiments can be combined with each other.

[0010] By the way, the program for realizing the software appearing in this embodiment may be provided as a non-transitory computer-readable medium that can be read by a computer, may be provided so as to be downloadable from an external server, or may be provided so as to realize its function on a client terminal by starting the program on an external computer (so-called cloud computing).

[0011] In addition, in the present embodiment, the "unit" may include, for example, hardware resources implemented by a circuit in a broad sense and information processing of software that can be specifically realized by these hardware resources. Also, in the present embodiment, various types of information are handled. These types of information are represented, for example, by physical values of signal values representing voltage and current, the high and low of signal values as a group of binary bits composed of 0 or 1, or quantum superposition (so-called quantum bits), and communication and calculation can be executed on a circuit in a broad sense.

[0012] Also, a circuit in a broad sense is a circuit realized by appropriately combining at least a circuit, circuitry, a processor, a memory, etc. That is, it includes an application specific integrated circuit (ASIC), programmable logic devices (for example, a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)), etc.

[0013] <Embodiment 1> 1. System Configuration Hereinafter, the system configuration according to Embodiment 1 will be described. FIG. 1 is a diagram showing an example of the overall configuration of the image generation system 1. In FIG. 1, each device included in the image generation system 1 and an overview of the users who use these devices are shown. Each overview will be described as needed with reference to other figures. The image generation system 1 is an information processing system that generates an image using an artificial intelligence module having a function of generating an image. The image generation system 1 executes a learning process for causing the artificial intelligence module to perform machine learning, an image generation process for causing the learned artificial intelligence module to generate an image, and the like.

[0014] The image generation system 1 includes a communication line 2, a server device 10, an operator terminal 20, and a user terminal 30. The communication line 2 is not particularly limited, but is configured by, for example, the Internet network. Further, the communication line 2 may include a local area network, a mobile communication network, a VPN (Virtual Private Network), and the like. The communication line 2 mediates the exchange of data between devices connected to its own line. In the example of FIG. 1, the server device 10 is wired-connected to the communication line 2, and the operator terminal 20 and the user terminal 30 are wirelessly connected. Note that the connection of each device to the communication line 2 may be wired or wireless.

[0015] The server device 10 is an information processing device that executes a learning process, an image generation process, and the like. The server device 10 stores an image database DB1, a dataset database DB2, and a learning model database DB3. The image database DB1 stores images and the like used for machine learning. The dataset database DB2 stores information (hereinafter referred to as "dataset information") regarding a dataset including a group of images selected from the images stored in the image database DB1.

[0016] The dataset mentioned here refers to a collection of data used as training data for machine learning. Dataset information is, for example, information that can identify images (hereinafter referred to as "DS images") included in the dataset. The learning model database DB3 stores information about learning models (hereinafter referred to as "learning model information") generated by machine learning performed using the dataset. The dataset information and the learning model information will be described in detail later.

[0017] The server device 10 is equipped with an image generation AI module 100. The image generation AI module 100 is an artificial intelligence module adjusted (tuned) to generate images using AI (Artificial Intelligence) technology. The image generation AI module 100 performs machine learning using a dataset including DS images specified by the dataset information stored in the dataset database DB2, and generates images having features similar to the DS images. Hereinafter, the images generated by the image generation AI module 100 are referred to as "AI images".

[0018] The operator terminal 20 is a terminal for an operator who operates the image generation system 1, and is, for example, a notebook computer or the like. The user terminal 30 is a terminal for an image user who uses the AI image on a web page or the like, and is, for example, a notebook computer or the like. The server device 10 executes display processing for displaying images on the operator terminal 20 and the user terminal 30, and authentication processing for authenticating the users of the operator terminal 20 and the user terminal 30.

[0019] The server device 10 performs processes such as generation and transmission of HTML (Hyper Text Markup Language) files as display processes, and causes a web page showing a system screen to be displayed on user terminals such as the operator terminal 20 and the user terminal 30. Note that the user terminal may introduce an application program for using the image generation system 1, and the server device 10 may perform processes such as generation and transmission of display data in the application as display processes. By performing these display processes, the server device 10 controls the display of the user terminal.

[0020] The server device 10 stores authentication information (such as a user ID and a password) for authenticating users such as operators and image users, and authenticates the user who has input the authentication information. By authenticating the user, the server device 10 can restrict access to data and make it possible to identify the data input by the user.

[0021] 2. Hardware Configuration Hereinafter, the hardware configuration according to Embodiment 1 will be described. FIG. 2 is a diagram showing an example of the hardware configuration of the server device 10. The server device 10 includes a control unit 11, a storage unit 12, a communication unit 13, and a bus 14. The bus 14 electrically connects each part included in the server device 10.

[0022] (Control Unit 11) The control unit 11 has at least one processor. The at least one processor may be constituted by, for example, a central processing unit (CPU), an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), one or more integrated circuits, one or more discrete circuits, and combinations thereof, which are not shown.

[0023] The control unit 11 is a computer that realizes various functions related to the image generation system 1 by reading a predetermined program stored in the storage unit 12. That is, the information processing by software stored in the storage unit 12 is specifically realized by the control unit 11, which is an example of hardware, and can be executed as each functional unit included in the control unit 11. Note that the control unit 11 is not limited to being single, and may be implemented to have a plurality of control units 11 for each function, or a combination thereof may also be possible.

[0024] (Storage unit 12) The storage unit 12 stores various information defined by the foregoing description. This can be implemented as a storage device such as a solid state drive (SSD) or HDD (Hard Disk Drive) that stores various programs and the like related to the image generation system 1 executed by the control unit 11, or as a memory such as a random access memory (RAM) that stores temporarily necessary information (arguments, arrays, etc.) related to the calculation of the program. The storage unit 12 stores various programs, variables, etc. related to the image generation system 1 executed by the control unit 11.

[0025] (Communication unit 13) The communication unit 13 is composed of a communication module. The communication module may be a wireless communication module compliant with standards such as IEEE802.11a / b / g / n / ac / ax, LTE, 5G, 6G, etc., or a wired communication module compliant with standards such as IEEE802.3. The communication unit 13 is configured to be able to transmit various electrical signals from the server device 10 to external components. Also, the communication unit 13 is configured to be able to receive various electrical signals from external components to the server device 10. More preferably, the communication unit 13 has a network communication function, and thereby various information may be communicated between the server device 10 and external devices via the communication line 2.

[0026] FIG. 3 is a diagram showing an example of the hardware configuration of the operator terminal 20. The operator terminal 20 includes a control unit 21, a storage unit 22, a communication unit 23, an input unit 24, an output unit 25, and a bus 26. The bus 26 electrically connects each part included in the operator terminal 20. The control unit 21, the storage unit 22, and the communication unit 23 may have different specifications, models, etc. from the control unit 11, the storage unit 12, and the communication unit 13 shown in FIG. 2, but they are the same hardware.

[0027] (Input Unit 24) The input unit 24 has keys, buttons, a touch screen, a mouse, etc., and receives inputs from the user. Also, the input unit 24 has a microphone and receives voice inputs from the user.

[0028] (Output Unit 25) The output unit 25 has a display, a speaker, etc., and displays visual information generated in a manner visible to the user, such as a screen, an image, an icon, text, etc. on the display surface of the display, and outputs sounds including voice.

[0029] The user terminal 30 shown in FIG. 3 has the same hardware configuration as the operator terminal 20. Regarding the user terminal 30, only the control unit 31 will be described with a different reference numeral from the control unit 21 of the operator terminal 20.

[0030] 3. Information Processing Hereinafter, the information processing according to the embodiment will be described. In the following description, the server device 10, the operator terminal 20, and the user terminal 30 are described as the main bodies of each information processing, but their information processing is executed by the processors included in the control units of the respective devices. The information processing executed by each processor includes, in addition to the learning processing and the image generation processing described above, a dataset creation process for creating a dataset used in machine learning.

[0031] FIG. 4 is an activity diagram showing an example of the dataset creation process. The dataset creation process shown in FIG. 4 starts when the user of the operator terminal 20 performs an operation to display a dataset creation screen. First, the server device 10 generates screen data showing the dataset creation screen and transmits the generated screen data to the operator terminal 20 (A11). The operator terminal 20 displays the dataset creation screen indicated by the transmitted screen data (A12).

[0032] On the dataset creation screen, a DS image (an image to be included in the dataset) is selected by the operation of the operator. As a method for selecting a DS image, there are methods such as individually inputting the file name and path name of the image to be selected as the DS image, or specifying the folder in which the image to be the DS image is stored. In the example of FIG. 5, a method of selecting an image to be the DS image after narrowing down images with common features from a large number of images as candidate images (hereinafter referred to as "candidate images") for the DS image will be described.

[0033] FIG. 5 is a diagram showing an example of the displayed dataset creation screen. On the dataset creation screen G1 shown in FIG. 5, a string "Please select an image to be included in the dataset.", an input field E11 for the image user, an input field E12 for the dataset name, an input field E13 for the image attribute condition, an input field E14 for the image right condition, an image display field E15, a select all button B11, a deselect button B12, and a confirm button B13 are displayed.

[0034] As described above, the person using the AI image is input as the image user in the input field E11. The name of the dataset to be created is input in the input field E12. Filtering conditions for narrowing down candidate images are input in the input fields E13 and E14. These filtering conditions are conditions that can be determined based on the information associated with each image in the image database DB1 described later.

[0035] In the input field E13, conditions for the attributes of candidate images are input as narrowing-down conditions. The image attribute conditions are, for example, the size, resolution, and subject of the image, etc. The subject refers to the main object shown in a photograph when the image is a photograph, and includes people, landscapes, etc. Note that an image with a "person" as the subject refers to an image in which it is possible to identify who the person is and for which the right of portrait occurs for that person. The attribute conditions listed here are just examples, and any attribute (for example, the date and time of shooting, the shooting location, or the name of the person who is the subject in the image, etc.) may be used as long as it is a condition for narrowing down candidate images.

[0036] For example, when "large", "medium", or "small" is input as the size of the image in the input field E13, only one size may be input, multiple sizes may be input, or a condition that does not narrow down the size, such as "all sizes", may be input. In this way, for the image attribute conditions, it is sufficient that one or more conditions are input. Note that when no conditions are input, the conditions may not be narrowed down in the same way as when "all sizes" is input.

[0037] In the input field E14, conditions regarding the rights that the image user has for candidate images are input as narrowing-down conditions for candidate images. The rights that the image user has are, for example, the copyright given to the person who created the candidate image, the copyright that the creator (author) of the work shown in the candidate image has, the right of portrait that the person shown in the candidate image can claim, or the right to use the candidate image (hereinafter simply referred to as "permission") permitted by the owner of those rights, etc.

[0038] In the input field E14, availability, purpose of use, available period, etc. are input. The availability indicates whether the image user can currently use the image, in other words, whether the image user currently has any right that enables the use of the image. Hereinafter, among the rights that enable the use of the image, the copyright or its permission will be referred to as "copyright-related rights", and the right of portrait or its permission will be referred to as "portrait-related rights".

[0039] In the following cases, when the subject is a person's image, if the image user has both copyright-related rights and portrait-related rights, it is marked as "○" indicating that it can be used; if the image user has only one of the copyright-related rights and portrait-related rights, it is marked as "△"; if the image user has neither copyright-related rights nor portrait-related rights, it is marked as "×". Also, when the subject is an image other than a person, if the image user has copyright-related rights, it is marked as "○" indicating that it can be used; if the image user does not have copyright-related rights, it is marked as "×". Note that even for images marked as "△" or "×", if they are images that the user wants to use for machine learning, it is possible for the image user to obtain copyright-related rights or portrait-related rights and use them, so they can be set as filtering conditions.

[0040] When the image user has copyright and portrait rights, there are no particular restrictions on the purpose of using the image. However, if the image user is a licensee who has obtained permission to use from the rights holder who retains those rights, the purpose of use may be restricted by the contract with the licensor. In the input field for the purpose of use, the purpose of use restricted in this way is entered. The purpose of use includes, for example, use on a web page, use in advertising, use within the company, and use outside the company.

[0041] When the image user has copyright and portrait rights, the user can use the image indefinitely as long as they have those rights. However, if the image user is a licensee, the end of the period during which use is permitted, that is, the expiration date of the period during which use is possible, may be determined by the contract with the licensor. In the input field for the expiration date of use, the expiration date during which the image user can use the image as determined in this way is entered. For example, if an expiration date of "until the end of 2025" is entered, the search will be narrowed down to images that can be used at least until the end of 2025.

[0042] In the input field E14, for example, as information indicating whether an image can be used, only "○" may be input, or multiple conditions such as "○" and "△" may be input, or a condition that does not narrow down the images based on usability such as "all" may be input. Thus, for the image right conditions, similar to the image attribute conditions, it is sufficient if one or more conditions are input. Note that when no conditions are input, the conditions may not be narrowed down in the same way as when "all" is input.

[0043] The operator terminal 20 receives (A13) the input of the narrowing-down conditions for candidate images by the user. The operator terminal 20 transmits the input narrowing-down conditions to the server device 10. The server device 10 reads out, as candidate images, the images that satisfy the transmitted narrowing-down conditions from the images stored in the image database DB1 (A14).

[0044] FIG. 6 is a diagram showing an example of the image database DB1. In the image database DB1, the image file name of each image, the attribute information of the image, and the contract information are stored in association with each other. The image file name may include, in addition to the file name of the image data, the path to the folder in which the image data is stored. As the attribute information of the image, the size, resolution, subject, etc. of the image are stored. The subjects of the images include people, landscapes, etc. Note that in addition to these, the subjects may include buildings, vehicles, animals, etc.

[0045] As the contract information of the image, the copyright owner of the image, the licensee of the copyright, the license conditions of the copyright, the owner of the portrait right, the licensee of the portrait right, the license conditions of the portrait right, etc. are stored. Note that when the subject is not a person, no portrait right occurs, so when the subject is a landscape or the like, the portrait right, etc. is "NA" (not applicable). The license conditions include the usage purpose of the image and the available period of the image described in FIG. 5.

[0046] In addition to these conditions, the license conditions may also include conditions for clearly defining the scope of copyrighted works and portraits in the image, restrictions on the modification of copyrighted works and portraits in the image, usage fees (including remuneration or royalties to the copyright owner and the portrait right owner), available period (including not only the expiration date of availability but also the start date of use), the obligation of the copyright owner and the portrait right owner to indicate, the obligation to protect the privacy and reputation of the portrait right owner, or conditions such as the available area. Further, the contract information may include information such as the name of the contracting party, the contract date or contract number, and the image data of the contract document. The server device 10 reads out images that meet the filtering conditions of these attribute information and contract information and transmits them to the operator terminal 20. The operator terminal 20 displays the transmitted images as candidate images (A15).

[0047] FIG. 7 is a diagram showing an example of the displayed candidate image. In FIG. 7, the dataset creation screen G1 shown in FIG. 5 is shown. In the example of FIG. 7, "Company C1" is input as the image user, and "DSv1.1 for Company C1" is input as the dataset name. The operator terminal 20 displays the transmitted candidate images in the image display column E15 shown in FIG. 5 and displays check boxes E16 in association with those candidate images. The candidate images displayed in the display column E15 are only a part, and it is assumed that other candidate images can also be displayed by an operation such as scrolling.

[0048] The operator checks the check box E16 of the candidate image to be included in the dataset. When the all-selection button B11 is operated, all the check boxes E16 are checked, and when the deselection button B12 is operated, the checks of all the check boxes E16 are canceled. Each time the operator changes the conditions input in the image attribute condition input column E13 and the image right condition input column E14, the operations from A13 to A15 are performed and the displayed candidate images are updated.

[0049] The user roughly filters candidate images according to filtering conditions, for example. After checking all the check boxes E16, the user unchecks the check boxes E16 of the unnecessary candidate images to select the candidate images to be included in the dataset. Note that there may be a function to display candidate images that do not meet the filtering conditions in another display area so that candidate images that do not meet the filtering conditions can be added. When the candidate images to be included in the dataset are determined, the user operates the determination button B13.

[0050] The user terminal 20 accepts the input operation of the filtering conditions, the check operation, and the operation on the determination button B13 as the selection operation and determination operation of the candidate images (A16), and transmits selection result data indicating the selection result to the server device 10. The server device 10 determines the candidate images indicated by the transmitted selection result data as DS images, and generates information about the dataset including the determined DS images as dataset information (A17). Then, the server device 10 stores and saves the generated dataset information in the dataset database DB2 shown in FIG. 1 (A18).

[0051] FIG. 8 is a diagram showing an example of the generated dataset information. In the dataset database DB2 shown in FIG. 8, the filtering conditions, the image information, and the right information are stored as dataset information in association with the dataset name. The filtering conditions store the conditions input in the input field E13 for the image attribute conditions and the input field E14 for the image right conditions on the dataset creation screen G1. Although not all the candidate images filtered by these filtering conditions are included in the dataset, the filtering conditions are stored as reference information representing the characteristics of the images included in the dataset.

[0052] The image information includes the image file name and information such as the subject. The image file name indicates the file names (path names may also be included) of all the DS images included in the dataset. The server device 10 can read out the DS images included in the dataset by referring to this image file name.

[0053] For the subject, the ratio of the subjects appearing in all DS images included in the dataset is shown. For example, in the case of the dataset "DSv1.1 for Company C1", the ratio of the subjects is shown such as 80% for people and 10% for landscapes. The server device 10 calculates this ratio by counting the number of subjects in all DS images stored in the image database DB1. Note that, in addition to the subject, information such as the number of images, image size, and resolution may be stored in the image information. In short, as long as the information representing the characteristics of those DS images is included as the image information so that it is easy to estimate what kind of DS images are included in the dataset.

[0054] The right information includes the image user (in the figure, "user"), availability information, copyright ownership, copyright license, its license conditions, portrait right ownership, portrait right license, and its license conditions, etc. The image user is the user of the DS image indicated by the right information.

[0055] The availability information is information indicating the availability of all DS images included in the dataset. As described above, the availability of a DS image is indicated as "○" when the image user has both copyright-related rights and portrait-related rights (only copyright-related rights when the subject does not include a person), "△" when having only one of them, and "×" when not having both of them (only copyright-related rights when the subject does not include a person). The availability information is indicated as "○" when all DS images are "○", "△" when there is at least one "△", and "×" when there is at least one "×".

[0056] "Copyright Ownership" indicates the percentage of images among all DS images that the image user owns the copyright for. "Copyright License" indicates the percentage of images among all DS images that the image user has been licensed to use the copyright for. "License Conditions" of copyright indicates the percentage of images among all DS images that have common license conditions. In the example of Figure 8, the percentage of images with a common available period among the license conditions is shown. In the case of the dataset "DSv1.1 for Company C1", it is shown as "Unlimited" being 50%, "~20xx" being 25%, and "~20yy" being 10%.

[0057] "Portrait Right Ownership" indicates the percentage of images among all DS images where the subject is a person that the image user owns the portrait right for. "Portrait Right License" indicates the percentage of images among all DS images where the subject is a person that the image user has been licensed to use the portrait right for. "License Conditions" of portrait right indicates the percentage of images among all DS images where the subject is a person that have common license conditions. In the example of Figure 8, the percentage of images with a common available period among the license conditions is shown. In the case of the dataset "DSv1.1 for Company C1", it is shown as "Unlimited" being 40%, "~20xx" being 30%, and "~20yy" being 15%.

[0058] The server device 10 calculates the above-mentioned right information by referring to the contract information of each DS image stored in the image database DB1, and stores it in the dataset database DB2. Note that the right information shown in Figure 8 is just an example and is not limited to these. The right information may include, for example, license conditions other than the available period, or only the license condition with the highest percentage of common images. In short, as long as the information representing the characteristics of the rights of those DS images is included as the right information so that it is easy to estimate the status of the rights of the DS images included in the dataset.

[0059] Subsequently, a learning process for performing machine learning using the dataset created by the dataset creation process described above will be explained. FIG. 9 is an activity diagram showing an example of a learning process. The learning process shown in FIG. 9 starts when the user of the operator terminal 20 performs an operation to display a machine learning screen. First, the server device 10 generates screen data indicating the machine learning screen, and transmits the generated screen data to the operator terminal 20 (A21). The operator terminal 20 displays the machine learning screen indicated by the transmitted screen data (A22).

[0060] FIG. 10 is a diagram showing an example of the displayed machine learning screen. On the machine learning screen G2 shown in FIG. 10, a character string "Please select a dataset to be used for machine learning.", an input field E21 for the image user, an input field E22 for the learning model name, a display field E23 for the list of datasets, and an execution button B21 for machine learning are displayed. In the input field E21, the person who uses the AI image is input as the image user. In the example of FIG. 10, "Company C1" is input as the image user.

[0061] When the operator terminal 20 receives the input of the image user (A23), it transmits user data indicating the input image user to the server device 10. When the user data is transmitted, the server device 10 refers to the dataset database DB2 and reads out the dataset information associated with the image user indicated by the user data (A24). The server device 10 transmits the read dataset information to the operator terminal 20. The operator terminal 20 displays the transmitted dataset information in the display field E23 of the list of datasets (A25).

[0062] In the example of FIG. 10, the operator terminal 20 displays the dataset name, availability ( "Availability" in the figure), subject, copyright license conditions, and portrait right license conditions as the dataset information. In the example of FIG. 10, the dataset information of the datasets created with "Company C1" as the image user, namely "DSv1.1 for Company C1", "DSv1.2 for Company C1", "DSv1.3 for Company C1", "DSv2.1 for Company C1", and "DSv2.2 for Company C1", is displayed.

[0063] Note that the dataset information displayed in the display column E23 is not limited to that shown in FIG. 10. For example, the number of images, resolution, ratio of copyright to its license, ratio of portrait right to its license, etc. may be displayed in the display column E23. In short, as a criterion for selecting a dataset used for machine learning, information representing the characteristics of the DS images and the characteristics of the rights included in the dataset may be displayed as dataset information.

[0064] The operator selects a dataset to be used for machine learning with reference to the displayed dataset information. In the example of FIG. 10, the operator has selected "DSv1.3 for Company C1". Note that in the example of FIG. 10, only one dataset is selected, but two or more datasets may be selected. The operator inputs the name of the learning model generated by machine learning. In the example of FIG. 10, the operator has input "Model v1.3 for Company C1". The operator operates the execution button B21 for machine learning to cause the image generation AI module 100 to execute machine learning on the selected dataset.

[0065] The operator terminal 20 accepts the above operations as an operation (A26) for selecting a dataset and an operation (A27) for instructing machine learning, and transmits instruction data indicating the selected dataset and the name of the input learning model to the server device 10. The server device 10 refers to the file name of the DS image included in the dataset indicated by the transmitted instruction data in the dataset database DB2, and reads out the image with the referred file name as a dataset (A31).

[0066] Next, the server device 10 instructs the image generation AI module 100 to execute machine learning using the read data set (A32). The image generation AI module 100 executes machine learning using the instructed data set (A33). As a result of executing machine learning, the image generation AI module 100 generates a learning model capable of outputting an image similar to the DS image included in the data set. The server device 10 stores and saves the generated learning model in the learning model database DB3 in association with the learning model information regarding the learning model (A34).

[0067] FIG. 11 is a diagram showing an example of the stored learning model information. In the learning model database DB3 shown in FIG. 11, the learning model information is stored in association with the learning model. The learning model information includes a data set name, image information, and right information. The data set name indicates the name of the data set used for machine learning when generating the learning model. For example, "Model v1.1 for Company C1" is associated with "DS v1.1 for Company C1". Also, "Model v1.2 for Company C1" is associated with two data sets, "DS v1.1 for Company C1" and "DS v2.1 for Company C1".

[0068] The image information and the right information read and stored are the image information and the right information stored in association with the corresponding data set in the data set database DB2. For example, "Model v1.1 for Company C1" is associated with image information including the ratio of subjects such as "Person: 80%", right information including the image user "Company C1", the availability "○", the license conditions of copyright such as "Unlimited: 50%", and the license conditions of portrait right such as "Unlimited: 40%".

[0069] The server device 10 refers to the image information and right information stored in the dataset database DB2, reads out the information associated with the dataset used for machine learning, and stores it in the learning model database DB3. Note that for a learning model with two or more datasets used for machine learning, such as "Model v1.2 for Company C1", the server device 10 refers to the image information and contract information associated with the images included in those datasets in the image database DB1, recalculates the ratio of the subject and the ratio of the license conditions, etc., and stores them as image information and right information.

[0070] The learning model information shown in FIG. 11 is an example and is not limited to these. The learning model information may include image information such as the number of images or the resolution, or may include only the license conditions other than the expiration date or the license conditions with the highest ratio of common images as the right information. In short, as long as the information representing the characteristics of the DS images included in the dataset used for machine learning of the learning model and the characteristics of the rights of those DS images is included as the learning model information, it is easy to estimate the characteristics of the DS images and the status of the rights of those DS images.

[0071] Subsequently, an image generation process for generating an AI image using the learning model generated by the learning process described above will be described. FIG. 12 is an activity diagram showing an example of the image generation process. The image generation process shown in FIG. 12 starts when the user of the operator terminal 20 performs an operation to display an image generation screen. First, the server device 10 generates screen data indicating the image generation screen, and transmits the generated screen data to the operator terminal 20 (A41). The operator terminal 20 displays the image generation screen indicated by the transmitted screen data (A42).

[0072] FIG. 13 is a diagram showing an example of a screen for generating a displayed image. In the image generation screen G3 shown in FIG. 13, a string “Please select a learning model to be used for image generation.”, an input field E31 for an image user, a display field E32 for a list of learning models, an input field E33 for a prompt, and an execution button B31 for image generation are displayed. In the input field E31, a person who uses an AI image is input as an image user. In the example of FIG. 13, “Company C1” is input as the image user.

[0073] When the operator terminal 20 receives the input of the image user (A43), it transmits user data indicating the input image user to the server device 10. When the user data is transmitted, the server device 10 reads out learning model information associated with the image user indicated by the user data (A44). The server device 10 transmits the read learning model information to the operator terminal 20. The operator terminal 20 displays the transmitted learning model information in the display field E32 of the list of learning models (A45).

[0074] In the example of FIG. 13, the operator terminal 20 displays the learning model name, availability ( “Available / Not Available” in the figure), subject, copyright license conditions, and portrait right license conditions as learning model information. In the example of FIG. 13, the learning model information of learning models generated by machine learning executed with Company C1 as the image user, namely “Model v1.1 for Company C1”, “Model v1.2 for Company C1”, “Model v1.3 for Company C1”, “Model v2.1 for Company C1” and “Model v2.2 for Company C1” is displayed.

[0075] Note that the learning model information displayed in the display field E32 is not limited to that shown in FIG. 13. For example, the number of images, resolution, ratio of copyright and its license, or ratio of portrait right and its license may be displayed in the display field E32. In short, as a criterion for selecting a learning model to be used for AI image generation, information representing the features of the DS images and the features of the rights included in the dataset used for the machine learning of the learning model may be displayed as the learning model information.

[0076] The operator selects a learning model to be used for generating AI images with reference to the displayed learning model information. In the example of FIG. 13, the operator has selected "Model v1.3 for Company C1". The operator inputs a sentence (so-called prompt) in the input field E33 of the prompt to instruct the AI image to be generated. Then, by operating the execution button B31 for image generation, the operator causes the image generation AI module 100 to generate an AI image using the selected learning model with the input prompt.

[0077] The operator terminal 20 receives the above operations as an operation (A46) for selecting a learning model, an operation (A47) for inputting a prompt, and an operation (A48) for instructing the generation of an AI image, and transmits instruction data indicating the selected learning model and the input prompt to the server device 10. The server device 10 executes an AI image generation process (A51) based on the transmitted instruction data. Specifically, the server device 10 executes a process of instructing the image generation AI module 100 to generate an AI image using the learning model and the prompt indicated by the instruction data. The image generation AI module 100 generates an AI image based on this instruction.

[0078] Next, the server device 10 reads out the learning model information associated with the learning model used for generating the AI image in the learning model database DB3 (A52). Subsequently, the server device 10 generates metadata of the generated AI image based on the read learning model information (A53). Then, the server device 10 generates screen data indicating an AI image screen for displaying the generated AI image and the metadata, and transmits the generated screen data to the operator terminal 20 (A54). The operator terminal 20 displays the AI image screen indicated by the transmitted screen data (A55).

[0079] FIG. 14 is a diagram showing an example of the displayed AI image screen. In the AI image screen G4 shown in FIG. 14, a string "AI image is completed.", a display column E41 for the AI image, a display column E42 for the metadata, a back button B41, and an image regeneration button B42 are displayed. In the display column E41, the generated AI image F41 is displayed. In the display column E42, the generated metadata M41 is displayed.

[0080] In the example of FIG. 14, the server device 10 generates metadata M41 including an "image ID", a "learning model name", a "model security", a "dataset name", a link L41 to the DS image, and a link L42 to the contract information. The "image ID" shows the identification information of the AI image issued by the image generation AI module 100 when generating the AI image F41. The "learning model name" shows the name of the learning model used for generating the AI image F41. The "dataset name" shows the name of the dataset used for the machine learning of the learning model used for generating the AI image F41.

[0081] The "model security" shows the legal security of the learning model used for generating the AI image F41. The server device 10 shows the "availability" included in the learning model information of the learning model used for generating the AI image F41 as the "model security". That is, the server device 10 shows "○" when both the copyright-related rights and the portrait-related rights are available for all the DS images included in the dataset used for the machine learning of the learning model used for generating the AI image F41, "△" when at least one image lacking one of them is included, and "×" when at least one image lacking both of them is included.

[0082] When the return button B41 is operated, the server device 10 returns to the image generation screen G3 shown in FIG. 13 of the previous screen and displays the screen immediately before the execution button B31 for image generation is operated. The operator can select the learning model again on the displayed image generation screen G3. When the image regeneration button B42 is operated, the server device 10 executes the processes after A51 again without changing the learning model or the like, and displays the newly generated AI image and metadata on the operator terminal 20.

[0083] The link L41 to the DS image indicates a link to an image included in the dataset indicated by "dataset name". When the link L41 is operated, the operator terminal 20 accepts the operation and transmits to the server device 10 the name of the dataset linked by the operated link L41 and instruction data for instructing the display of the DS image (A61). The server device 10 reads out the DS image included in the dataset with the name indicated by the transmitted instruction data from the image database DB1 (A62), and generates a DS image screen showing a list of the read DS images (A63). The server device 10 generates screen data showing the generated DS image screen, and transmits the generated screen data to the operator terminal 20. The operator terminal 20 displays the DS image screen indicated by the transmitted screen data (A64).

[0084] FIG. 15 is a diagram showing an example of the displayed DS image screen. In the DS image screen G5 shown in FIG. 15, a string "These are images included in the dataset used for machine learning.", a display column E51 for the image user, a display column E52 for the dataset name, a display column E53 for the image attribute conditions, a display column E54 for the image right conditions, and a display column E55 for the DS image are displayed. The server device 10 controls the display of these display columns with reference to the image database DB1, the dataset database DB2, and the learning model database DB3.

[0085] The server device 10 displays the name of the image user who uses the generated AI image in the display column E51, and displays the name of the dataset used for machine learning in the display column E52. Further, the server device 10 displays the filtering conditions (attribute conditions and right conditions) satisfied by the DS images used for machine learning in the display columns E53 and E54, and displays the DS images included in the dataset in the display column E55. The DS images displayed in the display column E55 are only a part, and it is assumed that other DS images can also be displayed by an operation such as scrolling.

[0086] FIG. 15 shows an example of the display of DS images when candidate images are filtered by the filtering conditions as described in FIG. 5 and the like and then selected. In the example of FIG. 15, by displaying the filtering conditions, it is possible to grasp what common features the DS images have. Note that when the selection of the DS image is performed by specifying a file name, a path name, or a folder name, those file names or the like may be displayed instead of the filtering conditions.

[0087] The link L42 to the contract information indicates a link to the contract information of the images included in the dataset indicated by the "dataset name". When the link L42 is operated, the operator terminal 20 receives the operation, and transmits the name of the dataset linked by the operated link L42 and instruction data for instructing the display of the contract information to the server device 10 (A71). The server device 10 reads out the contract information of the DS images included in the dataset with the name indicated by the transmitted instruction data from the image database DB1 (A72), and generates a contract information screen showing a list of the read contract information (A73). The server device 10 generates screen data showing the generated contract information screen, and transmits the generated screen data to the operator terminal 20. The operator terminal 20 displays the contract information screen indicated by the transmitted screen data (A74).

[0088] FIG. 16 is a diagram showing an example of the displayed contract information screen. In the contract information screen G6 shown in FIG. 16, a character string "This is contract information of images included in the dataset used for machine learning.", a display column E61 for image users, a display column E62 for dataset names, and a display column E63 for a list of contract information of DS images are displayed. The server device 10 controls the display of these display columns with reference to the image database DB1, the dataset database DB2, and the learning model database DB3.

[0089] The server device 10 displays the name of the image user who uses the generated AI image in the display column E61, and displays the name of the dataset used for machine learning in the display column E62. Further, the server device 10 displays in the display column E63 by associating the file names of the DS images used for machine learning and their contract information. The contract information includes the copyright owner of the image, the licensee of the copyright, the license conditions of the copyright, the owner of the portrait right, the licensee of the portrait right, and the license conditions of the portrait right, etc. For DS images whose subject is not a person, the portrait-related rights are indicated as "NA" (not applicable).

[0090] As described above, in the image generation system 1, the server device 10 functions as an example of an information acquisition unit that acquires related information related to the image group used for machine learning. The image group used for machine learning, in other words, is the image group included in the dataset, that is, a plurality of DS images. The server device 10 acquires learning model information as related information, for example, at A52 shown in FIG. 12. As shown in FIG. 11, the learning model information is information indicating the image information and the right information of a plurality of DS images. Thus, the related information (learning model information) acquired by the server device 10 includes the right information about the image group used for machine learning.

[0091] The right information indicates the right to use images (DS images) included in the image group used for machine learning. In the example of FIG. 11, the image user who is the subject of the right to use the DS image, the availability of using the DS image, the license conditions of copyright, the license conditions of the right of portrait, etc. are included in the right information. And the server device 10 functions as an example of an output unit that outputs metadata indicating the related information acquired about those image groups in association with the AI images generated by an artificial intelligence module (the image generation AI module 100 is an example thereof) that has performed machine learning using those image groups.

[0092] In the example of FIG. 14, the server device 10 outputs and displays the metadata M41 to the operator terminal 20 in association with the AI image F41. In the metadata M41, for the DS image used by the image generation AI module 100 that generated the AI image F41 for machine learning, that is, the DS image included in the data set "DSv1.3 for Company C1", the right information indicating that the "safety of the model" (the same information as the availability of using the DS image) is "○" is shown.

[0093] According to such an aspect, compared with the case where the metadata as shown in FIG. 14 is not output, it is possible to easily grasp the information about the image group used for machine learning, that is, the plurality of DS images used by the artificial intelligence module that generated the AI image for machine learning. In the example of FIG. 14, it is possible to easily grasp the right information about the plurality of DS images used for the machine learning of the artificial intelligence module in generating the AI image F41.

[0094] If, hypothetically, machine learning is performed using an image group that includes DS images for which the user of the AI image does not have the right to use, that is, an image group with deficiencies in rights, it may be determined that there was an intention to infringe on the rights of others, or it may be determined that due care was not taken to avoid infringing on the rights of others. On the contrary, if machine learning is performed using an image group that includes only DS images for which the user of the AI image has the right to use, that is, an image group without deficiencies in rights, it is likely to be recognized that there is no intention to infringe on the rights of others and that due care is being taken to avoid infringing on the rights of others.

[0095] By indicating the right information in the metadata, it is possible to know whether the artificial intelligence module that generated the AI image corresponding to the metadata performed machine learning using an image group with deficiencies in rights or an image group without deficiencies in rights. Therefore, the user of the AI image can grasp the legal security of the AI image.

[0096] Also, the server device 10 functions as an example of a selection reception unit that receives the selection of images (DS images) to be included in the image group used for machine learning. In the examples of FIGS. 5 and 7, the server device 10 receives the selection of DS images by displaying the dataset creation screen G1. As shown in FIG. 6, contract information representing the content of the contract that defines the right to use those DS images is associated with the selected DS images.

[0097] The server device 10 functions as an example of a generation unit that generates right information about the image group based on the contract information associated with the DS images included in the image group by the above selection. In the example of FIG. 8, for example, the server device 10 generates right information such as the ratio of DS images for which the availability and permission conditions of the DS images are common based on the contract information indicating the copyright-related rights and portrait-related rights associated with the DS images. According to such an aspect, the labor of preparing the right information can be reduced compared to the case where the user prepares the same right information manually.

[0098] In addition, the server device 10 functions as an example of an output unit that outputs metadata so as to be displayable to the user terminal. The operator terminal 20 is an example of a user terminal, and in the example of FIG. 14, the metadata M41 is being displayed. Then, when a display operation of an image group used for machine learning is performed on the user terminal on which the metadata is displayed, the server device 10 functions as an example of an image display unit that causes the user terminal to display an image included in the image group indicated by the related information based on the metadata. According to such an aspect, it is possible to facilitate the verification of the actually machine-learned images as compared with the case where the display control of the DS image by the above display operation is not performed.

[0099] In addition, when a display operation of the details of the rights is performed on the user terminal on which the metadata is displayed, the server device 10 functions as an example of a contract display unit that causes the user terminal to display the contract information that is the basis of the right information indicated by the metadata. In the example of FIG. 14, when an operation on the link L42 to the contract information included in the metadata M41 is performed as a display operation of the details of the rights to use the image group used for machine learning, the server device 10 causes the operator terminal 20 to display the contract information of the DS image in which the right information is indicated by the metadata M41 as shown in FIG. 16. According to such an aspect, it is possible to facilitate the verification of the contract content indicating the rights of the image as compared with the case where the display control of the contract information by the above display operation is not performed.

[0100] <Modification Example: Right Information> The right information included in the metadata is not limited to the information (model safety = availability) shown in FIG. 14. FIG. 17 is a diagram showing an example of the displayed metadata. In the example of FIG. 17, the server device 10 causes the metadata M42 to be displayed in the metadata display column E42 shown in FIG. 14. In addition to the right information (model safety) shown in FIG. 14, the metadata M42 includes "copyright-related rights" and "portrait-related rights".

[0101] Specifically, as for copyright-related rights, it includes "right of copyright = 4%" and "license of copyright = 96%" as the "source of use", "indefinite: 45%, ~20xx: 35%, ~20yy: 5%, ···" as the "usable period", and "web page, advertisement, in-house materials, external materials, ···" as the "purpose of use". Also, as for portrait-related rights, it includes "right of portrait = 2%" and "license of portrait right = 98%" as the "source of use", "~20xx: 30%, ~20yy: 25%, indefinite: 20%, ···" as the "usable period", and "web page, advertisement, in-house materials, external materials, ···" as the "purpose of use".

[0102] Thus, the rights information may include information indicating the conditions under which the right to use the DS image becomes effective. The conditions under which the right to use the DS image becomes effective are, in addition to the usable period and purpose of use shown in Fig. 17, conditions such as the usable area of the DS image, the renewal conditions of the license, or the license fee. By including these conditions in the metadata, it is possible to alert the user of the AI image to ensure that the right to use the DS image does not become invalid.

[0103] For example, if the user of the AI image is alerted that the DS image includes a usable period up to 20xx, the user can take measures such as setting the use of the generated AI image up to 20xx or extending the usable period of the corresponding DS image. Also, the user of the AI image can be careful not to use the AI image generated for purposes other than the permitted purposes of use.

[0104] In addition, the rights information may include information indicating the amount of DS images for which the conditions under which the right to use the DS image becomes effective are common. In the example of Fig. 17, the proportion of DS images with a common usable period is included in the rights information, such as "indefinite: 45%, ~20xx: 35%, ~20yy: 5%, ···". Note that the information indicating the amount of DS images is not limited to the proportion and may be represented by the number of images or the data size, etc. According to such a mode, it is possible to grasp the degree of influence in the case where the right to use becomes invalid for some DS images.

[0105] Also, when the conditions for the validity of the right to use the DS image are likely to be not satisfied, the server device 10 may display the right information indicating the conditions in a manner different from other right information. For example, when there is a DS image whose expiration date is approaching within one year, the server device 10 may display the expiration date in bold red or add a string such as "The expiration date is approaching within one year!" to prompt special attention. According to such a manner, it is possible to make it less likely that a situation where the right to use the DS image becomes invalid occurs compared to the case where only a uniform display is made.

[0106] Also, the right information may include information about the type of the right to use the DS image. The types of the right to use the DS image are, for example, the "copyright-related right" and the "portrait-related right" shown in FIG. 17. The copyright-related right is an example of the right regarding the person who created the DS image, and the portrait-related right is an example of the right regarding the person depicted in the DS image. Note that at least one of these rights may be included.

[0107] The rights regarding the person who created the DS image may include not only copyright but also, for example, design right or trademark right, etc. Also, the rights regarding the person depicted in the DS image may include publicity right or privacy right, etc. Even when any of those rights is included in the right information, it is possible to evaluate the security regarding the rights in detail compared to the case where only the availability of using the DS image is included in the right information.

[0108] In addition, the rights information may include information indicating the amount of DS images with common types of rights to use DS images. In the example of FIG. 17, the rights information includes the ratio of DS images with common types of rights, such as "Copyright = 4%" and "License of copyright = 96%". Note that the information indicating the amount of DS images is not limited to the ratio, and may be represented by the number of images, data size, etc. According to such an aspect, it is possible to grasp the balance of the types of rights and, if necessary, take measures to adjust the balance (for example, measures such as reducing the license ratio and increasing the ownership rate of copyright).

[0109] In addition, when the subject of the DS image is a person, the protected part by the licensed portrait right may be limited. For example, when the DS image is a close-up photo of a face, the AI image representing the face-up is likely to be protected by the rights licensed for the portrait right of the DS image, but the AI image representing the whole body is likely not to be protected by that right. Conversely, when the DS image is a full-body photo, the AI image representing the whole body is likely to be protected by the rights licensed for the portrait right of the DS image, but the AI image representing the face-up is likely not to be protected by that right.

[0110] Therefore, the rights information may include information indicating the scope of rights among the objects depicted in the images included in the image group. The "rights" mentioned here refer to the rights to use DS images, for example, the rights licensed from the portrait right owner. In this case, in the image database DB1, information indicating the scope of rights (hereinafter referred to as "scope information") is stored in association with each image. When the subject depicted in the DS image is a person, "face", "whole body", "upper body" or "sideways posture", etc. are stored as scope information.

[0111] When a DS image to be included in a dataset is selected, the server device 10 refers to the image database DB1, reads out the range information associated with those DS images, and generates right information based on the read range information. For example, the server device 10 generates, as the right information, information indicating the amount of DS images with a common scope of rights. Then, when generating an AI image, the server device 10 outputs metadata including the right information based on the range information.

[0112] FIG. 18 is a diagram showing an example of the displayed metadata. In the example of FIG. 18, the server device 10 causes the metadata M43 to be displayed in the metadata display column E42 shown in FIG. 14. In addition to the right information (model security) shown in FIG. 14, the metadata M43 includes, as "portrait-related rights", "scope of rights: face = 65%, upper body = 30%,...".

[0113] In this case, the user of the AI image may determine the composition of the AI image based on the right information indicated by the metadata. For example, in the example of FIG. 18, since the scope indicated by the right information, that is, the scope of rights permitted for the DS image, is substantially limited to the upper part of the upper body, when generating an AI image representing a person, the prompt may be used to instruct to generate an AI image with a composition that represents the upper part of the upper body larger. According to such an aspect, an AI image with a composition having less legal risk can be generated. In addition, by including, in the metadata, the right information indicating the amount of DS images with a common scope of rights, it is possible to intuitively understand a composition with less legal risk.

[0114] In addition to the above, the right information may also include information on items such as the name of the person entitled to use the DS image, the owner of the copyright-related rights and portrait-related rights, and the usage fee. It is desirable that the metadata includes, as the right information, information on items that are useful for the user of the AI image to understand the right situation of the DS image that is the source of the AI image, judge the legal risk, or take measures to avoid the risk.

[0115] In addition, the right information may include a plurality of items as in the examples of FIGS. 17 and 18. The plurality of items may desirably include two or more of the presence or absence of rights (an example thereof is "model safety"), the conditions for the rights to be valid (an example thereof is "usable period"), the types of rights (examples thereof are "copyright-related rights" and "portrait-related rights"), the scope of the rights, the types of sources of the rights (an example thereof is "source of use"), or the names of the rights. According to such an aspect, the right safety regarding the generated AI image can be evaluated comprehensively.

[0116] <Modification example: Selection of right information> The right information included in the metadata may be selectable. In that case, the server device 10 displays a selection screen for selecting the right information. Since it is conceivable that both the operator and the user of the AI image select the items of the right information, the selection screen may be displayed on either the operator terminal 20 or the user terminal 30.

[0117] FIG. 19 is a diagram showing an example of the selection screen of the displayed right information. On the selection screen G7 of the right information shown in FIG. 19, a check box E71 for the items of the right information to be selected, an image E72 of the metadata, and a decision button B71 are displayed. The check box E71 includes the items of the right information such as "presence or absence of rights", "type of rights", "name of rights", "usable period", "usable area", "usage fee", and "update conditions", and the check boxes thereof (indicating that "○" is checked).

[0118] The image E72 shows the image of the metadata when the items checked in the check box E71 are included. In the example of FIG. 19, "presence or absence of rights", "type of rights", "usable period", and "usage fee" are checked. In the image E72, "presence or absence of rights" is represented by the name "model safety", and "type of rights" is represented by the names "copyright-related rights" and "portrait-related rights". Also, "usable period" and "usage fee" are represented by the same names as the items.

[0119] The user (operator or user of the AI image) checks the items in the check box E71 while viewing the image E72. When the items to be included in the metadata are determined, the user operates the decision button B71. In this way, the server device 10 functions as an example of a selection display unit that causes the user terminal to display a selection image for receiving a selection operation of items to be included in the metadata among the plurality of items included in the right information. The operation of checking the check box E71 and the operation on the decision button B71 are examples of selection operations, and the selection screen G7 is an example of a selection image. Then, the server device 10 (an example of an output unit) outputs, as metadata, data indicating the items selected by the selection operation. According to such an aspect, the user can include desired items in the metadata.

[0120] <Modification Example: Modification of DS Image> The DS image included in the data set may be changed after the creation of the data set. In that case, in A11 and A12 of the data set creation process shown in FIG. 4, the server device 10 displays a change screen for changing the DS image instead of the data set creation screen. Since the DS image can be changed by either the operator or the user of the AI image, the change screen may be displayed on either the operator terminal 20 or the user terminal 30.

[0121] FIG. 20 is a diagram showing an example of a change screen for the displayed DS image. In the change screen G8 for the DS image shown in FIG. 20, a character string "Please reselect the image to be included in the data set.", an input field E81 for the image user, an input field E82 for the data set name, a display field E83 for the DS image, a check box E84, an input field E85 for the image attribute condition, an input field E86 for the image right condition, a display field E87 for the candidate image, a check box E88, an input field E89 for the data set name after the change, and a decision button B81 are displayed.

[0122] When an image user is input by the user in the input field E81, the name of the data set associated with the image user becomes inputtable in the input field E82. When a data set name is input in the input field E82, the DS images included in the data set are displayed in the display field E83. The displayed DS images are checked in the corresponding check box E84, but can be removed from the data set by unchecking.

[0123] In the input field E85 for image attribute conditions and the input field E86 for image right conditions, conditions for narrowing down candidate images to be added to the data set are input. When the narrowing-down conditions are input, the operations from A13 to A15 shown in FIG. 4 are performed, and candidate images that satisfy the narrowing-down conditions input by the user are displayed in the display field E87. Among the displayed candidate images, the candidate images for which the corresponding check box E88 is checked are added to the data set.

[0124] In the input field E89, the data set name after the DS image is changed is input. When the determination button B81 is operated in this state, the operations from A16 to A18 shown in FIG. 4 are performed, and data set information including the data set name input in the input field E89 is newly generated and saved. At this time, right information about the newly generated data set is stored in the data set database DB2 shown in FIG. 8.

[0125] In this way, the server device 10 functions as an example of a change reception unit that receives changes to the DS images included in the image group of the data set based on, for example, the user's operation on the change screen G8. And the server device 10 (an example of a generation unit) generates right information for the image group based on the contract information associated with the DS images included in the image group of the data set after the change is received. According to such an aspect, even after the data set is changed, the labor for preparing the right information can be reduced compared to the case where the user prepares the same right information manually.

[0126] Note that the server device 10 may automatically generate the name of the dataset after the change and display it in the input field E89. For example, when referring to the input field E82 and there is a part "v(N1).(N2)" (N1 and N2 are natural numbers) in the input dataset name, the server device 10 generates a dataset name "v(N1).(N2 + 1)" and displays it in the input field E89. For example, in the example of FIG. 20, the server device 10 generates a changed dataset name "DS for Company C1 v3.2" based on the dataset name "DS for Company C1 v3.1" and displays it in the input field E89.

[0127] Also, when the content of the change in the DS image satisfies the condition indicating a significant change, the server device 10 may generate a dataset name (v(N1 + 1), (N2)) with a major version upgrade, and when the condition is not satisfied, generate a dataset name (v(N1), (N2 + 1)) with a minor version upgrade. The condition indicating a significant change is, for example, a condition satisfied when a predetermined ratio (e.g., 30% or 50%) or more of the DS images included in the dataset has changed. According to such an aspect, the trouble of inputting the dataset name after the change can be saved.

[0128] <Modification Example: Component> The artificial intelligence module (e.g., the image generation AI module 100) may have a plurality of components. A component is a part that constitutes a part of the artificial intelligence module. The artificial intelligence module generates one AI image when a plurality of components operate respectively. These plurality of components perform machine learning using the same or different image groups respectively.

[0129] When the artificial intelligence module has the above plurality of components, the server device 10 stores learning model information different from the example of FIG. 11 in the learning model database DB3. FIG. 21 is a diagram showing another example of the stored learning model information. In the learning model database DB3 shown in FIG. 21, learning models such as "model ML1", "model ML2", and "model ML3" are stored.

[0130] Each learning model is associated with three components, "CMP1", "CMP2", and "CMP3", and the dataset names used for the machine learning of each component, respectively. In the example of FIG. 21, datasets DS1, DS2, and DS3 are used for machine learning. For example, in the case of "model ML1", DS1 is associated with CMP1, DS1 and DS2 are associated with CMP2, and DS3 is associated with CMP3.

[0131] Also, in the case of "model ML2", DS2 is associated with CMP1, DS3 is associated with CMP2, and DS2 is associated with CMP3. In the case of "model ML3", DS1 and DS2 are associated with CMP1 and CMP2, and DS2 and DS3 are associated with CMP3. Also, for each component, image information and right information corresponding to the dataset used for machine learning are associated. These image information and right information are information read from or recalculated from the dataset database DB2 as in the example of FIG. 11.

[0132] When the generation of an AI image is instructed, the server device 10 (an example of an output unit) outputs, as metadata, data indicating the related information acquired for the image groups used for machine learning by the respective plurality of components, in association with the AI image generated by the artificial intelligence module. The server device 10 outputs, for example, metadata indicating the right information for the image group included in the dataset used for machine learning as the related information, as in the example of FIG. 14.

[0133] FIG. 22 is a diagram showing another example of the displayed metadata. In the example of FIG. 22, the server device 10 causes the metadata display column E42 of the metadata shown in FIG. 14 to display the metadata M44. The metadata M44 includes a learning model name of "model ML1", component names of "CMP1", "CMP2", and "CMP3", dataset names of "DS1", "DS1+DS2", and "DS3", and the safety of all the models, all associated with "○".

[0134] In the example of FIG. 22, the user of the image can easily grasp the learning content (dataset) of each component compared to the case where the metadata is not output, and can easily grasp that each component is performing machine learning using DS images that are rightfully safe.

[0135] <Modification example: User-provided image> The images included in the dataset are images prepared in advance for the image generation system 1, but are not limited to this, and images provided by the user may be added. In that case, the server device 10 displays an additional screen for adding the images provided by the user (hereinafter referred to as "user images"). Since it is conceivable that both the operator and the user of the AI image can add the user image, the additional screen may be displayed on either the operator terminal 20 or the user terminal 30.

[0136] FIG. 23 is a diagram showing an example of the additional screen for the displayed user image. In the additional screen G9 for the user image shown in FIG. 23, a string "Please enter the user image to be added and its provided conditions.", an input field E91 for the file name of the user image to be added, an input field E92 for the user of the AI image, an input field E93 for the provided conditions regarding copyright, an input field E94 for the provided conditions regarding portrait rights, and a decision button B91 are displayed.

[0137] In the input field E91, in addition to the file name of the user image, a path name may also be input, and the file names of two or more user images may also be input. In the input field E92, the user who uses the AI image generated when the added user image is used as a DS image is input. The provision conditions input in the input fields E93 and E94 are the provision conditions for this user.

[0138] In the input field E93, provision conditions such as "copyright owner", "copyright transfer", "license", "usable period", "purpose of use", and "usage fee" are input. In "copyright owner", the name of the copyright owner of the user image, etc. is input. In "copyright transfer", "○" is input when the copyright of the user image is transferred to the image user. In "license", "○" is input when use is permitted without transferring the copyright. In "usable period", "purpose of use", "usage fee", etc., the conditions for permitting use are input.

[0139] In the input field E94, when there is a portrait right for the user image, provision conditions are input in the same way as in the input field E93. When each input field is input and the decision button B91 is operated, the server device 10 inquires of the image user whether or not to agree to the input provision conditions. When the server device 10 receives a reply from the image user agreeing to the provision conditions, it stores the user image in the image database DB1. At that time, the server device 10 stores the provision conditions input on the addition screen G9 in the image database DB1 as contract information.

[0140] The server device 10 stores, for example, the image user as the "copyright owner" in the image database DB1 when the copyright is transferred, and stores the input copyright owner when the copyright is not transferred. Note that the information such as the image user to be stored is a corporate name or a personal name. The server device 10 stores "NA" (not applicable) as the "licensee" when the copyright is transferred, and stores the image user when the copyright is licensed.

[0141] The server device 10 stores the provision conditions such as "usable period", "purpose of use", and "usage fee" as the "license conditions" of copyright. Also, when there are provision conditions for the right of portrait, the server device 10 similarly stores the "portrait right owner", "licensee of portrait right", and "license conditions" for the right of portrait. The provision conditions stored in the image database DB1 in this way are used as contract information for the dataset creation process shown in FIG. 4 and the like. As a result, the right information stored in the dataset database DB2 in the dataset creation process includes this provision condition.

[0142] As described above, the server device 10 functions as an example of an image acquisition unit that acquires a user image provided by the user. This user image is acquired based on the provision conditions agreed upon by the user and the image user as described above. And when the artificial intelligence module performs machine learning using the image group including the acquired user image, the server device 10 functions as an example of an information acquisition unit that acquires the information including the provision condition of the user image as the right information about the image group. According to such an aspect, since the user can include the image he / she likes as a DS image in the dataset, the tendency of the generated AI image can be made to suit the user's preference.

[0143] <Modification Example: Multiple Right Holders> The right holders of rights such as copyright and the right of portrait for the DS image may be multiple persons. For example, not only the photographer who took the DS image, but also the copyright owner of the work shown in the DS image may have the right to the DS image if there is a work in the DS image. Also, if multiple persons are shown in the DS image in a way that enables individual identification, those persons may all have the right of portrait.

[0144] In those cases, in the image database DB1 shown in FIG. 6, contract information about a plurality of rights holders is stored in association with one image respectively. Thereby, the server device 10 can narrow down candidate images to images to which a specific rights holder has rights by making it possible to input the rights holder as an image right condition on, for example, the dataset creation screen G1 shown in FIG. 5. Further, the server device 10 may also store dataset information and learning model information indicating the rights holder in the dataset database DB2 and the learning model database DB3 respectively.

[0145] By doing so, the server device 10 can display information about the rights holder of the DS image (for example, information indicating that the copyrights of the DS image are shared by Company C1 and Company C2, etc.) in the display column E23 of the list of datasets on the machine learning screen G2 shown in FIG. 10 and in the display column E32 of the list of learning models on the image generation screen G3 shown in FIG. 13, and can be used as a reference when selecting a dataset and a learning model.

[0146] <Modification Example: Relearning> The image generation AI module 100 may perform relearning in which machine learning is performed again using a learning model and a dataset (either a new dataset or a learned dataset) that have been generated once. In that case, the server device 10 stores the learning model used for relearning as a learning source model in the learning model database DB3.

[0147] FIG. 24 is a diagram showing another example of the stored learning model information. In the learning model database DB3a shown in FIG. 24, learning model information including the learning source model is stored. The learning source model shows the name of the learning model used when generating the corresponding learning model. In the example of FIG. 24, first, the “Model v1.1 for Company C1” generated first is not relearning, so only the dataset “DSv1 for Company C1” is associated, and the learning source model is “NA” (not applicable).

[0148] Using the "Model v1.1 for Company C1" as the original model, the trained model generated by retraining using the dataset "DS v2 for Company C1" is the "Model v1.2 for Company C1". Also, using the "Model v1.2 for Company C1" as the original model, the trained model generated by retraining using the dataset "DS v3 for Company C1" is the "Model v1.3 for Company C1", and using the "Model v1.2 for Company C1" as the original model, the trained model generated by retraining using the dataset "DS v4 for Company C1" is the "Model v2.1 for Company C1".

[0149] Furthermore, using the "Model v2.1 for Company C1" as the original model, the trained model generated by retraining using the dataset "DS v5 for Company C1" is the "Model v2.2 for Company C1", and using the "Model v1.3 for Company C1" as the original model, the trained model generated by retraining using the dataset "DS v4 for Company C1" is the "Model v3.1 for Company C1". Thus, according to the example in FIG. 24, it is possible to track the original model of the trained model that has undergone retraining over multiple generations. Note that in the example of FIG. 24, the management of the trained model information including the original model was performed in tabular form, but it may also be performed in node form.

[0150] FIG. 25 is a diagram showing an example of trained model information managed in node form. In the example of FIG. 25, the server device 10 stores nodes Nm1 to Nm6 indicating trained models and nodes Nd1 to Nd5 indicating datasets. Nodes Nm1 to Nm6 respectively indicate "v1.1", "v1.2", "v1.3", "v2.1", "v2.2", and "v3.1" of the "Model for Company C1". Nodes Nd1 to Nd5 respectively indicate "v1", "v2", "v3", "v4", and "v5" of the "DS for Company C1". Rights information corresponding to each dataset is associated with nodes Nd1 to Nd5. Note that trained model information corresponding to each trained model may be associated with nodes Nm1 to Nm6.

[0151] The server device 10 stores connection information indicating the connection of each node. In the example of FIG. 25, the connection of each learning model by re-learning shown in FIG. 24 is shown. For example, the connection information indicates that the node Nm2 (model v1.2 for company C1) is a learning model generated by re-learning using the node Nd2 (DS v2 for company C1) with the node Nm1 (model v1.1 for company C1) as the learning source model.

[0152] Also, the node Nm2 branches into a node Nm3 (model v1.3 for company C1) with a minor version upgrade and a node Nm4 (model v2.1 for company C1) with a major version upgrade. The connection information indicates that only the node Nm5 (model v3.1 for company C1) with a major version upgrade is connected from the node Nm3. The server device 10 may be able to display an image visually representing the relationship of the nodes as shown in FIG. 25 based on the information of each node and the connection information. Thereby, the relationship between the learning model, the learning source model, and the dataset can be made intuitively understandable.

[0153] <Modification Example: Related Information> The related information related to the image group used for machine learning is not limited to the right information (information indicating the right to use the DS image). For example, the related information may be the version information of the dataset or the image information regarding the DS image. The image information of the DS image includes the number of DS images included in the dataset, the resolution of the DS image, or the subject of the DS image, etc. Regarding the image information such as the resolution of the DS image and the subject of the DS image, it may be information indicating the amount of DS images with common image information.

[0154] <Modification Example: DS Image Browsing Function> In the example of FIG. 15, an example of the function of viewing the DS image from the metadata has been described, but the following aspects may also be applicable. The AI image and the metadata are, for example, viewed by the image user, and among the image users, there may be those who have the right to view, edit, or access the DS image (hereinafter referred to as "viewing rights, etc.") and those who do not.

[0155] In that case, the server device 10 may output the metadata including the link L41 to the DS image shown in FIG. 15 to the user terminal used by those who have the viewing rights, etc. of the DS image, and output the metadata not including the link L41 to the DS image to the user terminal used by those who do not have the viewing rights, etc. of the DS image. According to such an aspect, it is possible to prevent unauthorized persons from viewing the DS image.

[0156] Also, in the DS image screen G5 shown in FIG. 15, the number of DS images to be displayed may be extremely large. In that case, even if a simple list of DS images is displayed, it is difficult to verify the DS images. Therefore, the server device 10 may calculate, for example, feature amounts (such as HOG (Histograms of Oriented Gradients) or SIFT (Scaled Invariance Feature Transform)) used in image recognition for the DS images included in the dataset, and preferentially display representative images extracted from similar images within the range where the calculated feature amounts are common. According to such an aspect, it is possible to compare DS images with different features in a short time as compared with the case of displaying DS images regardless of the features of the images.

[0157] <Modification Example: Contract Information Viewing Function> In the example of FIG. 16, an example of the function of viewing contract information from metadata has been described, but the following modes may also be applicable. Similar to the DS image described above, the server device 10 outputs metadata including a link L42 to the contract information shown in FIG. 16 to the user terminal used by a person having the viewing right of the contract information, and outputs metadata not including the link L42 to the contract information to the user terminal used by a person not having the viewing right of the contract information. According to such a mode, it is possible to prevent a person without permission from viewing the contract information.

[0158] <Variation Example: Configuration Variation> The configuration (overall configuration, hardware configuration, functional configuration, etc.) shown in FIG. 1 and the like is an example, and other configurations can be adopted as long as there is no inconvenience in implementation. For example, the server device 10 may be distributed among two or more devices, or may be provided in the form of SaaS (Software as a Service) or a cloud computing system. Further, the information processing executed by the server device 10 may be collectively executed by a user terminal such as the operator terminal 20. In short, as long as the information processing necessary for the entire image generation system 1 is executed, the devices that execute these information processes may have any configuration.

[0159] The association between pieces of information may be performed in various modes. For example, the pieces of information may be associated by storing each piece of information in corresponding areas in a database or a table, or a table or the like showing the association between the pieces of information may be created for the association. Further, the association may be performed by adding information for identifying the other piece of information to one piece of information. In addition, the association may be performed by various well-known methods. The output destination of the information or data (hereinafter referred to as "information etc.") may be another device, a display, a storage unit (including a built-in storage unit and an external storage unit), or the like. The acquisition of information etc. includes a mode of acquiring information etc. transmitted from another device, in addition to a mode of acquiring information etc. generated by the own device.

[0160] The aspects of the above-described embodiments were information processing apparatuses such as the server apparatus 10, the operator terminal 20, and the user terminal 30, and an information processing system such as the image generation system 1 including those information processing apparatuses, but may also be an information processing method. The information processing method includes the same steps as those executed by the information processing system. Further, the aspects of the above-described embodiments may also be a program. The program causes a computer to execute the same steps as those executed by the information processing system.

[0161] <Supplementary Note> Furthermore, it may be provided in each of the aspects described below.

[0162] (1) An information processing system including a processor, wherein in the information acquisition step, the processor acquires related information related to an image group used for machine learning, and in the output step, metadata indicating the related information acquired for the image group is output in association with an image generated by an artificial intelligence module that performs machine learning using the image group.

[0163] According to such an aspect, it is possible to easily grasp information about the image group used for machine learning.

[0164] (2) The information processing system according to (1) above, wherein the related information includes rights information about the image group, and the rights information is information indicating the right to use the images included in the image group.

[0165] According to such an aspect, it is possible to grasp the legal security of the generated images.

[0166] (3) The information processing system according to (2) above, wherein the rights information includes information indicating conditions under which the rights are valid.

[0167] According to such an aspect, it is possible to alert about the expiration of rights.

[0168] (4) In the information processing system according to (2) or (3) above, the rights information includes information about the type of the right, and the type of the right includes at least one of a right related to the person who created the image and a right related to the person represented in the image.

[0169] According to such an aspect, the security regarding the right can be evaluated in detail.

[0170] (5) In the information processing system according to any one of (2) to (4) above, the rights information includes information indicating the scope to which the right extends among the objects depicted in the images included in the image group.

[0171] According to such an aspect, an image with a composition having less legal risk can be generated.

[0172] (6) In the information processing system according to any one of (2) to (5) above, the rights information includes a plurality of items, and the plurality of items include two or more of the presence or absence of the right, the conditions for the right to be valid, the type of the right, the scope to which the right extends, the type of the origin of the right, or the name of the right.

[0173] According to such an aspect, the legal security can be evaluated comprehensively.

[0174] (7) In the information processing system according to any one of (2) to (6) above, the rights information includes a plurality of items, and in the selection display step, the processor causes the user terminal to display a selection image for receiving a selection operation of an item to be included in the metadata among the plurality of items, and in the output step, outputs data indicating the item selected by the selection operation as the metadata.

[0175] According to such an aspect, desired items can be included in the metadata.

[0176] (8) In the information processing system according to any one of (2) to (7) above, in the selection reception step, the processor receives a selection of an image to be included in the image group, and contract information representing the content of a contract defining the right to use the image is associated with the image. In the generation step, the processor generates the right information for the image group based on the contract information associated with the image included in the image group by the selection.

[0177] According to such an aspect, the labor for preparing the right information can be reduced.

[0178] (9) In the information processing system according to (8) above, in the change reception step, the processor receives a change to an image to be included in the image group, and in the generation step, the processor generates the right information for the image group based on the contract information associated with the image included in the image group after the change is received.

[0179] According to such an aspect, the labor for preparing the right information can be reduced even after the dataset is changed.

[0180] (10) In the information processing system according to (8) or (9) above, in the output step, the processor outputs the metadata so as to be displayable to the user terminal. In the contract display step, when a display operation of the details of the right is performed on the user terminal on which the metadata is displayed, the processor causes the user terminal to display the contract information that is the source of the right information indicated by the metadata.

[0181] According to such an aspect, the verification of the contract content indicating the right of the image can be facilitated.

[0182] (11) In the information processing system according to any one of (1) to (10) above, in the output step, the processor outputs the metadata so as to be displayable to the user terminal, and in the image display step, when a display operation on the image group is performed on the user terminal on which the metadata is displayed, the processor causes the user terminal to display an image included in the image group indicated by the relevant information by the metadata. Information processing system.

[0183] According to such an aspect, it is possible to easily verify the learned actual images.

[0184] (12) In the information processing system according to any one of (1) to (11) above, the artificial intelligence module has a plurality of components, each of the plurality of components performs machine learning using the same or different image groups, and the artificial intelligence module generates one image by each of the plurality of components operating, and in the output step, the processor outputs, as the metadata, data indicating the relevant information acquired about the image groups used by the plurality of components for machine learning, respectively, in association with the image generated by the artificial intelligence module. Information processing system.

[0185] According to such an aspect, it is possible to easily grasp the learning contents of each component.

[0186] (13) In the information processing system according to any one of (2) to (10) above, in the image acquisition step, the processor acquires a user image provided by the user, the user image is acquired based on the provided conditions agreed by the user, and in the information acquisition step, when the artificial intelligence module performs machine learning using the image group including the acquired user image, the processor acquires information including the provided conditions of the user image as the right information about the image group. Information processing system.

[0187] According to such an aspect, it is possible to make the tendency of the generated images conform to the user preference.

[0188] (14) A program that causes a computer to execute each step executed by the information processing system according to any one of (1) to (13) above.

[0189] According to such an aspect, information about the image group used in machine learning can be easily grasped.

[0190] (15) An information processing method in which a processor included in an information processing system executes each step executed by the information processing system according to any one of (1) to (13) above.

[0191] According to such an aspect, information about the image group used in machine learning can be easily grasped. Of course, this is not the case. In addition, the above-described embodiments and modifications may be arbitrarily combined and implemented.

[0192] Finally, although various embodiments of the present invention have been described, these are presented as examples and are not intended to limit the scope of the invention. The novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the gist of the invention. The embodiments and their modifications are included in the scope and gist of the invention and are also included in the invention described in the claims and the equivalent scope thereof.

Explanation of Reference Numerals

[0193] 1: Image generation system 2: Communication line 10: Server device 11: Control unit 20: Operator terminal 21: Control unit 30: User terminal 31: Control unit 100: Image generation AI module DB1: Image Database DB2: Dataset Database DB3: Learning Model Database

Claims

1. An information processing system comprising a processor, wherein the processor in an information acquisition step, acquires related information related to an image group used for machine learning, in an output step, outputs metadata indicating the related information acquired for the image group, in association with an image generated by an artificial intelligence module that has performed machine learning using the image group, the information processing system.

2. In the information processing system according to Claim 1, the related information includes rights information about the image group, the rights information is information indicating the right to use the images included in the image group, the information processing system.

3. In the information processing system according to Claim 2, the rights information includes information indicating the conditions under which the rights become effective, the information processing system.

4. In the information processing system according to Claim 2, the rights information includes information about the type of the rights, the type of the rights includes at least one of the rights related to the person who created the image and the rights related to the person represented in the image, the information processing system.

5. In the information processing system according to Claim 2, the rights information includes information indicating the scope to which the rights extend among the objects depicted in the images included in the image group, the information processing system.

6. In the information processing system according to Claim 2, the rights information includes a plurality of items, the plurality of items includes two or more items among the presence or absence of the rights, the conditions under which the rights become effective, the type of the rights, the scope to which the rights extend, the type of the origin of the rights, or the name of the rights, the information processing system.

7. In the information processing system according to Claim 2, the rights information includes a plurality of items, the processor in a selection display step, causes a user terminal to display a selection image for receiving a selection operation of an item to be included in the metadata among the plurality of items, in the output step, outputs data indicating the item selected by the selection operation as the metadata, the information processing system.

8. In the information processing system according to Claim 2, the processor in a selection reception step, receives a selection of an image to be included in the image group, and contract information representing the content of a contract defining the right to use the image is associated with the image, In the generation step, based on the contract information associated with the images included in the image group by the selection, generate the right information for the image group. Information processing system.

9. In the information processing system according to claim 8, The processor is In the change reception step, receive a change to the images included in the image group. In the generation step, based on the contract information associated with the images included in the image group after the change is received, generate the right information for the image group. Information processing system.

10. In the information processing system according to claim 8, The processor is In the output step, output the metadata so as to be displayable to the user terminal. In the contract display step, when a display operation of the details of the rights is performed on the user terminal on which the metadata is displayed, display the contract information that is the source of the right information indicated by the metadata on the user terminal. Information processing system.

11. In the information processing system according to claim 1, The processor is In the output step, output the metadata so as to be displayable to the user terminal. In the image display step, when a display operation of the image group is performed on the user terminal on which the metadata is displayed, display the images included in the image group indicated by the related information by the metadata on the user terminal. Information processing system.

12. In the information processing system according to claim 1, The artificial intelligence module has a plurality of components. The plurality of components each perform machine learning using the same or different image groups. The artificial intelligence module generates one image by each of the plurality of components operating. The processor is In the output step, as the metadata, output data indicating the related information obtained for the image groups used by the plurality of components for machine learning, respectively, in association with the image generated by the artificial intelligence module. Information processing system.

13. In the information processing system according to claim 2, The processor is In the image acquisition step, acquire a user image provided by the user. The user image is acquired based on the provided conditions consented to by the user. In the information acquisition step, when the artificial intelligence module performs machine learning using an image group including the acquired user image, information including the provision conditions of the user image is acquired as the right information about the image group. Information processing system.

14. A program, A program that causes a computer to execute each step executed by the information processing system according to any one of Claims 1 to 13.

15. An information processing method, wherein a processor included in the information processing system executes each step executed by the information processing system according to any one of Claims 1 to 13. Information processing method.

Images